Code and comments

Practical and theoretical aspects of software development

Python 3 Library updates, via twitter

I’ve launched a new twitter account, @Py3K_update, which will send out tweets to notify followers when Python packages support Python 3. If this sounds like information that you want, please follow and/or tweet about this account.

If you are curious about the details, read on.

In February of 2011, Ubershmekel created the Python Wall of Shame, with hopes of motivating more migration towards Python 3. I saw the site, and thought it was interesting. A few days later I returned to the site, and quickly realized that it didn’t give me the information that I really wanted. I couldn’t look at a list of 200 Python packages on various days, and remember what had changed.

Since I’m not currently writing production Python 3 code, the question of which packages support Python 3 isn’t essential to me. But the larger question of the state of Python 3 adoption is interesting to me, and I wanted data that could give any objective insight into the progress that is being made.

So I wrote a Python script to scrape the data from the Wall of Shame (itself obtained by scraping PyPI), store it in a SQLite3 database, compare it to the previous information, and send tweets announcing libraries that are newly Python 3 compliant. The script is executed daily by cron. You can find the code at github.

So if you follow this account, you can expect to receive tweets shortly after libraries change from red to green on the Wall of Shame. With only 34 libraries currently supporting Python 3 of the top 200 packages from PyPI, I’m hoping that there will be a few tweets every month. I’ll probably add a monthly tweet to avoid complete silence in the case that progress is slow for a time.

Why twitter, anyway? Because it’s the absolute simplest, fastest, cheapest way to publish information, and because the data that I’m providing fits twitter perfectly, as it consists of short, public, messages that have transient value.

As this was my first non-trivial Python project, I learned a lot. For example:

  • How do you parse HTML in Python? (lxml.html)
  • How do you send a tweet in Python? (Python Twitter Tools)
  • How do you run all of your unit tests at once in Python? (nosetests)
  • How do you load data from a configuration file in Python? (ConfigParser)

in addition to some mundane things, like logging in Python, Python DB-API, learning some things about git and github, my first experience with cron, etc. Who knows, perhaps some of these will lead to another blog post.

Learning more Python was certainly my larger motivation, but I’m hoping that others in the Python world will appreciate the information provided by @Py3K_update.


Written by Eric Wilson

September 6, 2011 at 11:00 am

Posted in projects

Tagged with , , , ,

5 Responses

Subscribe to comments with RSS.

  1. How flattering and awesome indeed!


    September 6, 2011 at 12:16 pm

    • Glad you appreciate it … come to think of it, I probably should have asked permission.

      Eric Wilson

      September 6, 2011 at 8:03 pm

  2. I doubt anyone but me notices this, but there should have been a tweet today, as a library (I don’t know which one) apparently became Python 3 compliant within the last 24 hours, according to the Wall of Shame.

    The reason it didn’t happen was that I decided to change the time of the cron job from 8 AM to 5 AM for my convenience (allowing me to read the log file in the morning.) The problem was that I made that change at about 6:30 AM, guaranteeing that cron would not run the script today. As I was driving to work, I realized my error.

    I’ll kick the job off tonight, since I don’t have access to my computer remotely … which needs to be remedied …

    Eric Wilson

    September 7, 2011 at 1:19 pm

  3. Wow. Pretty neat. Scraping PyPl and such. Nice to see you are bending the power of programming to meet your needs.

    Jeff Ewer

    September 8, 2011 at 4:17 pm

  4. […] Python 3 Library updates, via twitter ::: Code and comments […]

Comments are closed.

%d bloggers like this: