Python

Programming Crutch, Litmus Test, or Silver Bullet?

Ryan Van Slooten — Tue, 23 Oct 2007 20:58:37 GMT

Lately I've been reading more about Ruby on Rails -- actually it has been rather entertaining, colorful, testy, and somewhat enlightening. One important aspect of programming is the ability to analyze, understand, and adapt. I have been using more Python lately in work as well as JavaScript and it has been enjoyable although I still like and prefer C#.

Sometimes I wonder how many programmers get fixated on specific language features that in turn becomes a "must have" or requirement for any language he or she uses. That isn't to say the feature is not important but rather usually the task can be accomplished in multiple ways albeit with more creativity (or even hacks) than otherwise necessary.

The examples can be found in any language or product comparison from C++/C#/Python/Ruby/Java to Oracle/SQL Server/MySQL and so on. I recently had to write a stored procedure on Oracle that manipulated data in a table and exported the data. I struggled for several hours as I am not intimately familiar with Oracle before deciding I could accomplish the same task in SQL Server in ten minutes. Sure enough, I used a linked server connection and a temporary table in SQL Server and had the issue solved in practically no time. The issue for me wasn't that Oracle could not accomplish the task, but that I was more familiar with SQL Server and the programming options I could use with it. Oracle does indeed have advantages over SQL Server and vice versa, but it seems IMHO that temporary tables are indeed easier to use in SQL Server and can be contained much easier (I initially tried local temporary tables in Oracle and then global temporary tables and then objects and tables of objects before going back to SQL Server). Ultimately, it was personal preference and familiarity though and not inability of the database. But we as programmers usually fall back on the same techniques that we are familiar with at times, rather than objectively using the best language or technique. It is somewhat like a sore spot that every time you rub against it becomes sorer (even when you shouldn't or don't need to touch that spot). On the other hand, we should be careful were preference out weighs objectivity.

When I read comments about one language being better than another, I always take the comments with a grain of salt. I like to think of programming language differences very similar to spoken and written language differences: the main point can almost always be expressed in another language although it may be longer or more verbose or less "natural" and may not convey the subtleties.

Finally, about Twitter having to write a small portion of code in C -- all of my projects have tight schedules and I would not be happy if I was the developer of the application. Even if it only took 300 lines of C and 2 hours to write, how much more time was spent analyzing, debugging, and getting to that point? And of course, it was solved by the authors of the platform who know it better than anyone else -- so it takes someone who wrote the system or at least very intimately familiar with the system to solve -- it seems to be a very small resource pool, which IMHO is a big problem. It is usually is easier to solve when you wrote the software and know where to look.

Subtext Configuration

Ryan Van Slooten — Thu, 03 May 2007 14:32:25 GMT

I really like Subtext, but unfortunately I haven't had too much time to try to configure my blog. I've looked through the wiki, but haven't seen enough information to emulate the configuration of Subtext founder Phil Haack. For example, the FeedBurner plugin which lists the number of subscribers and links to get the RSS and even email feeds -- how exactly is that configured? I suppose I should RTFM but I'm having trouble finding it at the moment (not to mention I have way too much to do right now with work!). It seems the Subtext Wiki is describing plug-in features in Subtext 2.0 (I'm running 1.9.x currently -- 2.0 is not released yet) so that doesn't help.

Sooner or later I will install Phalanger and test out DokuWiki. I also want to install umbraco and possibly Rainbow portal while I'm at it. I'd love to have GNU Mailman installed to manage mailing lists, but there are too many issues related its dependence on UNIX-style usernames and groups and I would probably need to user IronPython for the Python functionality. I guess it is just too much work right now, especially as I am utterly swamped at work. Unfortunately at work I am stuck working on a design portion of a project and I don't even get to do "real" programming that much -- sometimes you long to launch Visual Studio or work on source code. Oh well, hopefully this phase will end soon.

Altering SQL Server

Ryan Van Slooten — Wed, 04 Apr 2007 21:17:53 GMT

I was working on a project last week where I needed to update the columns and data types in tables in SQL Server. It is pretty quick to do this manually such as:

ALTER TABLE orders
    ALTER COLUMN orderid varchar(36) NOT NULL

I prefer this compared to the code SQL Server generates in the change script which typically renames the existing table, creates a new table, copies the data from the previous table to the new table, and then restores indices and keys. While that is appropriate for dramatic changes, it is overkill for something of this nature.

Since I love using tools and scripting, I decided to make a small Python program which would let me just specify which tables and columns to modify. This simplifies maintenance as I only have to manage the tables and columns rather than reading the entire SQL script looking for missing columns or keys.

While I was developing the Python script, I wanted to add enough SQL to avoid obvious errors such as trying to add duplicate primary keys or trying to alter columns that are already NOT NULL. I used MSDN and looked at some of the generated SQL to get the following templates:

IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID('orders') AND COLUMNPROPERTY(id, 'orderid', 'AllowsNull') = 1)
BEGIN
    PRINT 'Altering column: orders.orderid'
    ALTER TABLE orders
        ALTER COLUMN orderid varchar(36) NOT NULL
END
GO

IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID('orders') AND OBJECTPROPERTY(id, 'TableHasPrimaryKey') = 0)
BEGIN
    PRINT 'Adding primary key: orders.orderid'
    ALTER TABLE orders
        ADD CONSTRAINT PK_orders
            PRIMARY KEY (orderid)
END
GO

So the Python script used the string templates to write the output SQL and it looked like:

#!/usr/bin/env python

"""Generate a T-SQL script to set the appropriate NOT NULL
columns and PRIMARY KEY indices for the database.

TODO: Create indices for specific columns as well
"""

import string

# database properties

tables = {
    "orders":           { 'notnull': [('orderid', 'varchar(36)')],
                          'pk'     : ['orderid'] },

    "accounts":         { 'notnull': [('accountid', 'varchar(50)'),
                                      ('name', 'varchar(100)')],
                          'pk'     : ['accountid'] },

    # More tables and columns...
    }

# Functions

def generatenotnull(table, column, datatype):
    print alternotnull % { 'table':table, 'column':column, 'datatype':datatype }

def generatepk(table, columns):
    print alterpk % { 'table':table, 'columns':string.join(columns, '"') }

# SQL statements and formats

alternotnull = """IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID('%(table)s') AND COLUMNPROPERTY(id, '%(column)s', 'AllowsNull') = 1)
BEGIN
    PRINT 'Altering nullable column: %(table)s.%(column)s'
    ALTER TABLE %(table)s
        ALTER COLUMN %(column)s %(datatype)s NOT NULL
END
GO"""

alterpk = """IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID('%(table)s') AND OBJECTPROPERTY(id, 'TableHasPrimaryKey') = 0)
BEGIN
    PRINT 'Adding primary key: %(table)s.%(columns)s'
    ALTER TABLE %(table)s
        ADD CONSTRAINT PK_%(table)s
            PRIMARY KEY (%(columns)s)
END
GO"""


# Main entry
if __name__ == "__main__":
    for table in tables:
        print "/* ==== Processing %s ==== */" % table
        tabledict = tables[table]
        for column in tabledict['notnull']:
            generatenotnull(table, column[0], column[1])
        # TODO: Allow for compound primary keys
        generatepk(table, tabledict['pk'])
        print ""

Decisions (so many tools)

Ryan Van Slooten — Mon, 26 Mar 2007 14:21:52 GMT

During a database schema upgrade yesterday, I had to create a new primary key field for a table and then generate the new primary key values. At the moment this key value is a custom generated value by a Python class, but it will eventually use an RFC 4122 or ISO/IEC 9834-8 compliant algorithm (see Wikipedia for more information).

I had to decide how to generate this value and I ended up using a combination of Python, gawk, and T-SQL. I suppose I could have left out the gawk, but it would have taken almost the same amount of time to do it in Python and it took a just moment to write. I will present three different methods that I could have accomplished the same task, although there were reasons that I used the method that I did.

Method 1 - Python, gawk, and SQL

# Requires Python 2.5 or higher
import uuid
for i in range(200):
    print uuid.uuid4()

C:\>python genuuid.py > uuid.txt

C:\>gawk "/^'/ { i += 1; print \"INSERT @issueid VALUES (\" $1 \", \" i \")\"; }" uuid.txt &gt; uuid.sql

/* Add the table variable to the SQL file */
DECLARE @issueid TABLE (
    issueid varchar(36),
    issuenum int
)

Method 2 - Python and SQL

# Requires Python 2.5 or higher
import uuid

print """DECLARE @issueid TABLE (
    issueid varchar(36),
    issuenum int
)"""

for i in range(200):
    print "INSERT @issueid VALUES ('%s', %d)" % (uuid.uuid4(), i+1)

C:\>python genuuid.py > uuid.sql

Method 3 - SQL

/* Add the table variable to the SQL script */
DECLARE @issueid TABLE (
    issueid varchar(36),
    issuenum int
)
DECLARE @i int

SET @i = 0
WHILE @i < 200
BEGIN
    INSERT @issueid VALUES(NEWID(), @i+1)
    SET @i = @i + 1
END

/* Process table here */

After looking back, Method 2 seems to be the most straight-forward and self-contained method. The reason I chose Option 1 was that it was the first method that I thought of. Now that I am using Python more, I will probably start favoring quick solutions like Method 2.