Posted on October 29, 2008 14:13 by swilliams

I've been playing around with a few Python based web frameworks for the last few months in order to improve my non .NET based skills. What I like about all of them is that in terms of functionality and programmer friendliness they hold their own, if not surpass, more traditional "enterprise" frameworks.

The three I've spent the most time with are web.py, Pylons, and Django. I wouldn't hesitate to recommend them to anyone, but each has their own strengths and weaknesses.

web.py -- The Good

The calling card for web.py is:

"Think about the ideal way to write a web app. Write the code to make it happen."

and that's a succinct way to describe it. The syntax is very minimal, but expressive. The footprint is the smallest of the three I've used; I wrote the least amount of code out of the three of these frameworks with web.py, and that is a very good thing.

I love the way it handles routing and URL definition. A class represents a view, and GET and POST methods handle those particular http calls. This makes the most sense to me.

The built in template system was a little quirky, but easy enough once I got the hang of it. Installing and running in a development environment was also easy. All things considered, I would have been content using it exclusively, but for a few important issues:

web.py -- The Bad

First and foremost, the documentation is "sparse." Aaron Swartz (the creator and maintainer) is very smart and knowledgeable in Python, but I am not, and often could not figure out a solution to a problem without extensive research.

Installation on a production environment was much more difficult. I have only a basic knowledge of Linux administration, and was only able to get a web.py app running under the slower old CGI based way. Again, the documentation didn't really provide a lot of guidance beyond the basics.

web.py -- The Followup

It looks like some extra documentation has been created for web.py since I last took a look at it, sepcifically with regard to getting it up and running on a server. It does apply to version 0.3, but looks like it warrants another look.

Pylons -- The Good

My next stop on my Python Framework World Tour was Pylons. I found it because at some point reddit switched from web.py to this. I found Pylons to be much more newbie friendly. The documentation ranges from decent to good, and the "getting started" videos were incredibly valuable. Development was just as easy as web.py, though the dev server did not work on my Vista x64 workstation (this may be a Python issue though).

Pylons 0.9.6 uses the Mako template system, which probably was my favorite by a nose. There was a bit of a learning curve to it, but with ample documentation, I didn't have too much trouble.

Debugging Pylons is simple as well. If a page crashes, it presents the stack trace, but goes a step further and lets you execute code at each step in the stack. Just be sure to disable this in production.

The URL routing is great, though a little confusing.

Pylons -- The Bad

While not as sparse as web.py, Pylons' documentation could be better. Too often I found tutorials written for older versions that did not work any more. This next point may be a product of the prior one, but I also found the wiring of the models and controllers to be rather unintuitive and required too much code.

The biggest issue I had again came with deployment. I was unfamiliar with .egg packages and had to stumble around a bit before things were working. I ended up using FastCGI to serve pages, which also required to frustrating moments, but ultimately worked.

Despite these issues, I still maintain a simple photo blog built on top of Pylons.

Django -- The Good

Django is one of the biggest rising stars out there right now. It recently released version 1.0.

What I liked most about Django was the documentation. This is how you are supposed to do it. Thorough, but not overwhelmingly wordy, very quick, and well laid out.

The built in "admin" functionality is also incredibly useful. I wrote my own [crummy] version for the photo blog I mentioned above, but Django's would have been perfect had I known about it. This is an example of one of the many useful features thrown in.

I like the way Models are connected to the Persistance layer as well. You define a class, and the database structures are generated automatically, which makes the most sense to me.

Django does not allow in page debugging like Pylons, but does have a specialized Python shell that let's you perform all of the operations your page would.

Django -- The Bad

Of course, nothing is without fault. The biggest issue I have with Django is the URL routing. Both web.py and Pylons allow you to define a generic view to point to like this:

map.connect(':controller/:action/:id')

While Django basically forces you to be explicit while defining your routes. To me this seems to be too much, though others disagree that the generic way is more "Pythonic."

The template system takes a little time to get used to. It has a very small subset of functionality built in to it, and does not allow for inline python code. Rather, you use what are called filters to do things. You can develop your own filters as well pretty easily.

Memory seems to be an issue as well. My web server does not have a ton of RAM, which didn't seem to be a problem with web.py or Pylons, but when some of my django apps get busy, it quickly runs out of room. This may be developer error on my part though.

In Conclusion

I don't think I can give a recommendation to any one platform, all three in my opinion are strong enough for using in a modern app. web.py is for those who feel adventerous, and know python better than me. Pylons is a good middle ground and has a slightly better template system. Django is the easiest to set up and manage, and has the best documentation when you're stuck.



Digg It!DZone It!StumbleUponTechnoratiRedditDel.icio.usNewsVineFurlBlinkList

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Posted on March 16, 2008 00:34 by jpager

Two days ago, I posted an example of a way to apply IronPython scripting in .NET. At the very end of my post, I wondered about the possibility of compiling the script as a .NET assembly.

I'm happy to say, I have found a way to implement this functionality (and I might mention at this point that this is my last post about IronPython). First, I'll explain how I was able to do it, and then I'll explore whether we really gain anything from having a pre-compiled script.

First, compiling the script. The idea is that we only want to recompile our script if (a) the DLL does not exist or (b) our script is more recent than the DLL (i.e., it has been edited). We'll examine our files using the System.IO.FileInfo class. If we need to compile the script, we'll use IronPython's implementation of System.CodeDom.Compiler.CodeDomProvider, which will allow us to compile a python script into MSIL, and save the assembly as a DLL (this is located in the IronPython.CodeDom namespace):

FileInfo fi_in = new FileInfo(exportScript);
FileInfo fi_out = new FileInfo("BatchExport.dll");
if (!fi_out.Exists || (fi_in.LastWriteTimeUtc >= fi_out.LastWriteTimeUtc))
{
   PythonProvider p = new PythonProvider();
   CompilerParameters _params = new CompilerParameters(new string[] { }, "BatchExport.dll");
   CompilerResults r = p.CompileAssemblyFromFile(_params, new string[] { exportScript });
   if (r.Errors.Count > 0) throw new Exception("Compiler error.");
}


This is fairly straightforward. We create a compiler, call CompileAssemblyFromFile(), which is configured to save the assembly as BatchExport.dll and compile the file containing our python script. The compilation takes a short amount of time, so we only want to compile if absolutely necessary.

The next step is actually calling into the compiled assembly. As I mentioned last time, calling a Python assembly from C# is non-trivial. That python assembly contains dynamic types, which are not easily reflected into useful C# objects. So my approach was to create a PythonEngine, import the assembly into said PythonEngine, and call the method using Python.

To do this, I first altered my script so that my Export() method is contained in a class called BatchExport, rather than being a global method. Then, in my C# code, I created an instance of the BatchExport class, and called the Export() method on that object.

using (PythonEngine engine = new PythonEngine())
{
   engine.AddToPath(AppDomain.CurrentDomain.BaseDirectory);
   engine.Import("clr");
   engine.Execute("clr.AddReference('BatchExport.dll')");
   engine.Execute("import BatchExport");
   engine.Execute("be = BatchExport()");
   List<string> _params = new List<string>();
   _params.Add("order");
   ExportDelegate exMethod = engine.CreateMethod<ExportDelegate>("return be.Export(order)", _params);
   bool res = exMethod(order);
}


Our first step was to actually load our assembly into our python engine. To do that, we call clr.AddReference(). This allows us to include any .NET assembly in our script.

Then, we instantiated our BatchExport object, and use the CreateMethod() method to actually call the object's Export() method. Now the compile-script-only-if-it-has-changed functionality works perfectly.

Of course, after doing this, I made a big mistake--I decided to see how much time this actually buys us. So I created a simple script that processes a batch 500 times using the old method (that simply calls ExecuteFile()) and the same batch 500 times using the new method (I also created a new Export() python script that doesn't write to any files, in order to avoid any bias involving file sizes). The result was at least a little surprising--it took the method that pre-compiles the script 15 times as long to process the batches as it does the method that executes the script anew each time.

Possible culprits? Probably the cost of reflection is the main one. I don't think the check on the file times could make that big a difference. Clearly this little operation is quite expensive:

engine.Import("clr");
engine.Execute("clr.AddReference('BatchExport.dll')");
engine.Execute("import BatchExport");

So, I guess what we can take out of this is that compiling python scripts to .NET DLLs comes at a considerable cost, both in performance and complexity, at least if the script is simple. It may be cheaper to precompile the assembly if our script is hundreds (or thousands) of lines long, but for fairly basic scripts, you might as well just leave them as scripts.



Digg It!DZone It!StumbleUponTechnoratiRedditDel.icio.usNewsVineFurlBlinkList

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Posted on March 14, 2008 01:07 by jpager

Last week I explored the PythonEngine object in IronPython. We didn't get any deeper than simply using the API--this week I figured I'd actually put it to use in a real application.

Here is our scenario. You are a developer for a software company that writes a Point of Sale application on the .NET platform. You have hundreds of clients, each with their own in-house IT departments. You'll install everything for them, provide some support when needed, but you simply don't have time to offer each client major customizations. The problem is, each client has its own billing system--some legacy, some more modern--and there is no way for your development team to write a module to interface with each individual system. So, our approach will be to use scripting for the export module. This will allow our clients' in-house development teams to make changes to the export module on their own (or even rewrite the whole thing to their hearts' desire!) without having to even recompile the application. And, you guessed it, we'll use Python for our scripting (the solution and the scripts are attached).

Our first step is to actually build our point of sale application. Sparing all the technical details, I've created a simple Windows Form with a list of products, some fields for quantity, totals, etc., and a "Process" button. When the button is clicked, it will use our python script to send the order out to the billing system. To make things even more customizable, I'll also create an app.config file to store the location of the export python script.

Actually loading the python script is trivial--even more trivial than anything we discussed last week. We need nothing more than a call to the PythonEngine's ExecuteFile() method:

public static void Process(Batch order)
{
   // Do some stuff... Save to database, logs, etc.
   // Send outbound
   string exportScript = ConfigurationManager.AppSettings["ExportScript"];
   using (PythonEngine engine = new PythonEngine())
   {
      engine.ExecuteFile(exportScript);
      bool res = (bool)((IronPython.Runtime.Calls.PythonFunction)engine
         .Globals["Export"]).Call(order);
   }
}

And viola, we're finished with our C# code. We can package this and send it our for our clients. Just to be nice, we'll also provide two simple export scripts. One will be a flat file format that will be dropped off for a legacy system to pick up, and the other will generate simple XML for a more modern system.

Here is our first script:

def Export(batch):
   f = file('export.txt', 'a')
   f.write('BATCH BEGIN [[')
   f.write(str(batch.SubTotal) + '|' + str(batch.Tax) + '|' + str(batch.Total) + '\r\n')
   for item in batch.Items:
      f.write(item.Description + '\r\n')
      f.write('BATCH END ]]\r\n')
   f.close()
   return True

When we run this a couple of times, we get the following output:

BATCH BEGIN [[1.0|0.0850|1.0850 
Mountain Dew 
Mountain Dew 
BATCH END ]] 
BATCH BEGIN [[56.99|4.84415|61.83415 
Mountain Dew 
Mountain Dew 
Arrested Development (DVD) (entire series) 
BATCH END ]]   

And we'll also provide a script that exports it in a simple XML format:

def endl(file):
   file.write('\r\n')
def Export(batch):
   f = file('export.xml', 'a')
   f.write('<batch ')
   f.write('subtotal="'+str(batch.SubTotal)+'" ')
   f.write('tax="'+str(batch.Tax)+'" ')
   f.write('total="'+str(batch.Total)+'">')
   endl(f)
   if(len(batch.Items) == 0):
      f.write('\t<items />')
      endl(f)
   else:
      f.write('\t<items>')
      endl(f)
      for item in batch.Items:
         f.write('\t\t<item description="'+item.Description+'" />')
         endl(f)
      f.write('\t</items>')
      endl(f)
   f.write('</batch>')
   endl(f)
   f.close()
   return True  

This gives us the following output:

<batch subtotal="42.0" tax="3.5700" total="45.5700">
   <items>
       <item description="DVD Player" />
      <item description="Novelty Frisbee" />
      <item description="Novelty Frisbee" />
      <item description="Novelty Frisbee" />
      <item description="Novelty Frisbee" />
   </items>
</batch>
<batch subtotal="19.75" tax="1.67875" total="21.42875">
   <items>
      <item description="Novelty Frisbee" />
      <item description="Novelty Frisbee" />
      <item description="Novelty Frisbee" />
      <item description="Nebraska Replica License Plate" />
      <item description="Erick Dampier jersey (XXL)" />
      <item description="Erick Dampier jersey (XXL)" />
   </items>
</batch>    

And these are just two of the possibilities. Our clients can interact with their external system in any way imaginable--TCP, Message Queue, Web Service, Database--anything that is possible with .NET or python can be done.

Our last consideration is one of efficiency. It would be nice if we could save our compiled script in an assembly and use the pre-compiled version as long as the script is not updated. IronPython allows us to easily create a DLL with our python script using the IronPython.CodeDom namespace, but it is difficult to invoke the compiled code in C#, if it is even possible. A possible workaround may be to write a python script that references the assembly and invoke that script instead. I'll let you know if I ever get that to work.

PythonInterface.zip (15.66 kb)



Digg It!DZone It!StumbleUponTechnoratiRedditDel.icio.usNewsVineFurlBlinkList

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Posted on March 10, 2008 14:45 by swilliams

Recently stumbled across some pretty neat implementations of the [in]famous FizzBuzz question in one line. I saw a python based solution, but felt it could be tweaked a little bit more:

print "\n".join(["%s" % ['fizzbuzz','fizz','buzz',i][[i%15,i%3,i%5,0].index(0)] for i in range(1,101)])

This is very clever, and not terribly hard to understand once you know the syntax. Let's break it down into its parts.

First, is the "\n".join(). Python has a join method on strings that inserts the string as the delineator of a list. Coming from a C#/Java background this seems backwards, but hey, it works.

Moving along is the list that is joined by the newline. ["%s" % expression for i in range(1,101)] means that a list of strings will be created for each integer from 1 to 101, and the value will be determined by 'expression'.

['fizzbuzz','fizz','buzz',i] is the list of the potential objects to print, according to the rules of FizzBuzz. i is the integer that will be printed if we are not going to print the other three strings.

[[i%15,i%3,i%5,0].index(0)] is where the decision to print a word is made. It creates a list on the fly to see if the current integer is a multiple of 15, 3, or 5. index(0) then gets the first position that '0' occurs. This tells the prior list which index to use. So:

When i = 1, [i%15,i%3,i%5,0].index(0) becomes [1,1,1,0].index(0), which will return 3. Then ['fizzbuzz','fizz','buzz',i][3] is 'i', which in turn prints "1".

When i = 3, [i%15,i%3,i%5,0].index(0) becomes [3,0,3,0].index(0), which will return 1 (the first position of '0'). Then ['fizzbuzz','fizz','buzz',i][1] is 'fizz', which will be printed.

What I like about this is that it is a "legitimate" one liner. Yes, you could cook up something similar in C# with gratuitous semicolons, but that violates the essence of style that C# has.


Python Bites are little bits of Python code that have helped me out in my day to day. I wouldn't necessarily build an enterprise level application out of it, but it is usually quicker for smaller scripting tasks.



Digg It!DZone It!StumbleUponTechnoratiRedditDel.icio.usNewsVineFurlBlinkList

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Posted on March 7, 2008 22:44 by jpager

I don't know about you, but I am a fan of python. It has the advantages of flexibility that come from weakly typed languages, but maintains many of the functions of typical object-oriented languages. Though it feels like a scripting language, you can even write fully-featured standalone applications if you feel so inclined. So, naturally, I thought it would be fun to explore the possibility of using Python with .NET.

The de-facto standard .NET implementation of Python is IronPython. IronPython is a Python implementation that is written in .NET, and has use of all of the .NET CLR functionality. This allows us a couple of possibilities. We can use it as a python interpreter that has access to .NET functionality (for example, Windows Forms). However, as Scott has pointed out, if you're looking for a Python interpreter, you might as well use the native one, and if you're writing a .NET application, you might as well use C#. The other way we can use IronPython is purely as a scripting language that can be embedded in a C# (or VB, if you swing that way) application. (Particular ways to use scripting in .NET, while perhaps the most practical aspect of this discussion, will be reserved for some other time. My focus will be nothing more than using IronPython in a C# application.) So, without further ado...

Using IronPython in C#

At the center of the IronPython module is the PythonEngine object. So, the first thing we need to do is import our IronPython dll and create our PythonEngine object. Let's start by importing the IronPython.Hosting namespace:

using IronPython.Hosting;  

We create our object like so:

     using (PythonEngine engine = new PythonEngine()) {
     }  

Now that we have our PythonEngine, we can explore some of its basic functions.

The most basic function of PythonEngine is the Evaluate() method. Evaluate simple executes a single Python expression (note: by this I really mean expression, not statement. If you try an assignment, it will complain about bad python syntax.) and returns the result as an object (there is also a strongly-typed version, EvaluateAs, which I will use in my examples).

Let's try approximating PI using a python expression:

     double pi = engine.EvaluateAs<double>("4/1.0 - 4/3.0 + 4/5.0 - 4/7.0 + 4/9.0 "
          + "- 4/11.0 + 4/13.0 - 4/15.0 + 4/17.0 - 4/19.0 + 4/21.0 - 4/23.0 + 4/25.0 "
          + "- 4/27.0 + 4/29.0 - 4/31.0 + 4/33.0 - 4/35.0 + 4/37.0 - 4/39.0 + 4/41.0 "
          + "- 4/43.0 + 4/45.0 - 4/47.0 + 4/49.0 - 4/51.0");   

When we check the value of our pi variable, we get the value 3.1031453128860127, just as we expect.

Obviously, our value of pi is a little off, so we probably want to use the value of pi in python's math module. So, let's change our expression to this:

     double pi2 = engine.EvaluateAs<double>("math.pi"); 

If you try this, you'll get a runtime error: PythonNameErrorException: name 'math' not defined (and yes, the Exception class is "PythonNameErrorException"). This brings us to our next feature of the PythonEngine: Import. To get this to work, we need to import the math module, like so:

     engine.Import("math");
     double pi2 = engine.EvaluateAs<double>("math.pi"); 

Now, as we would expect, our pi2 variable now contains the value 3.1415926535897931. If you want to use your native python implementation's modules, you need to add those modules' path to your PythonEngine before it could find them:

     engine.AddToPath("C:\\Python25\\Lib");  

Let's move on to something a little more complicated: CreateMethod<TDelegate>(). CreateMethod is a strongly-typed function that returns a delegate. So, obviously, we need to define our delegate, with a contract that the function must adhere to:

     public delegate string DoSomethingDelegate (string param); 

CreateMethod has four overloads, but the one we'll deal with is the one that takes a python statement as the first parameter and a list of type names (the function signature) as the second:

     List<string> _params = new List<string>(new string[]{"param"});  
     DoSomethingDelegate function = engine.CreateMethod<DoSomethingDelegate>(@"
return 'Did you say \'' + param + '?\''  
          ", _params); 
     string rtn = function("TEST"); 

The last feature we'll look at is PythonEngine's Execute method. Execute is pretty self-explanatory--it simply executes Python code. The difference between Execute and the previous functions we've discussed is that Execute does not return anything--it only has side effects. It is important to understand that the PythonEngine works similarly to a native Python shell: any names that are defined while executing code are stored in the engine for future use. That is, if you define a name when you call Execute(), that name will be available to every subsequent Execute() call. Here is a simple example:

      engine.Execute(@"
def Foo():
     return 'Bar'
          ");
     string foo = engine.EvaluateAs<string>("Foo()"); 

In our execute() call, we defined a function called Foo(), and then referenced that same function when we called EvaluateAs().

Next, we'll actually call one of our Python functions from C# code. All of the global names in our PythonEngine are publicly accessible through the PythonEngine's Globals property. Any name that is a function is represented by the C# type IronPython.Runtime.Calls.PythonFunction, which has a method called, aptly enough, Call():

      engine.AddToPath("C:\\Python25\\Lib");
     engine.Import("os");
     engine.Execute(@" 
def RunCommand(command):
     p = os.popen(command)
     output = p.read()
     return output
          ");
     string ipconfig = (string)((IronPython.Runtime.Calls.PythonFunction)engine.Globals["RunCommand"])
          .Call(new object[] { "ipconfig" }); 

This does nothing more than create a process with the command sent through the parameter (which in this case is ipconfig) and read the output stream of the process. Because it uses the native Python os module, I added the path to that module to the PythonEngine's path. After this code executes, the ipconfig variable contains the output of the Windows ipconfig utility.

Now, let's put this to use and create something with it. We'll build a simple web form containing a panel, a text box, and a button. When the button is clicked, the command in the text box will be executed, and the output displayed in the panel (Disclaimer: Whatever you do, do not deploy this page to a network facing web site, for reasons that should be obvious to you).

Here's a snippet of our .aspx page:

     <asp:Panel ID="pnlOutput" runat="server" />
     <asp:TextBox ID="txtCommand" runat="server" Width="600" />
     <asp:Button ID="btSubmit" Text="Execute" runat= server  /> 

Here's our button's event handler:

     void btSubmit_Click(object sender, EventArgs e)
     {
          using (PythonEngine engine = new PythonEngine())
          {
               engine.AddToPath("C:\\Python25\\Lib");
               engine.Import("os");
               engine.Execute(@"
def Execute(command):
     pipe = os.popen(command)
     return pipe.read()
                    ");
               PythonFunction command = (PythonFunction)engine.Globals["Execute"];
               string res = (string)command.Call(new object[] { txtCommand.Text });
               pnlOutput.Controls.Add(new LiteralControl("<pre>" + res + "</pre>"));
          }
     } 

And there you have it. Try it out and play with it; we've only barely touched the possibilities.



Digg It!DZone It!StumbleUponTechnoratiRedditDel.icio.usNewsVineFurlBlinkList

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Posted on March 5, 2008 16:02 by swilliams

I needed to create an easy to use file of state abbreviations and names. I found something close from the USPS, but the format would have been a pain to parse in my application. So I decided to use Python to rearrange the text into something a little more sensible:

f = file('c:\users\swilliams\documents\code\states.txt', 'r')
for line in f:
	print line[-3:-1] + ',' + line[:-3].strip()
f.close()

And it worked perfectly. Not bad for 4 lines of code.


Python Bites are little bits of Python code that have helped me out in my day to day. I wouldn't necessarily build an enterprise level application out of it, but it is usually quicker for smaller scripting tasks.



Digg It!DZone It!StumbleUponTechnoratiRedditDel.icio.usNewsVineFurlBlinkList

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5