Addicted to Django's NewForms

NewForms is one of my favorite feature in Django, and I am getting addicted to it. The first thing that comes our mind on the newforms' usage is for adding or updating records. Being a newforms addict, I also use it for doing search and even for forms that has one input element (not including the input type = submit)

When doing search, i need the inputs to be validate and that's easy with newforms. Just choose appropriate field if the input needs to be a number, a date, and etc. I am really happy with what MultipleChoiceField provide to make my search more flexible.

Item.objects.filter(status__in=form.cleaned_data['status'])

Sometimes a simple form as having a one input element needs a complicated validation. With newforms you can do almost do anything you can with python. It's every extensible and I'll try to show you on my next post.

Hi! I am Firebird SQL

I've started using Firebird SQL (1 & 1.5) since 2001 mainly with Delphi. The combination is great especially with IBObjects. Before Firebird, I've use MS SQL 7, Interbase 6, and MySQL. I really had a great and wonderful experience with firebird compare to the rest that I've use.

Firebird comes with a lot of ANSI SQL features that's comparable to MS SQL, but not as complicated as MS SQL is. Programming triggers and store procedure is so much easier and it helps you manipulates your data. While MySQL is easy and lite,Firebird is also easy and lite without sacrificing many of the common important ANSI SQL features.

Presently, if I compare Firebird to MySQL and PostgreSQL, Firebird is somewhere in the middle of MySQL and PostgreSQL in terms of their respective strengths. One thing I don't understand is why Firebird doesn't get the attention/support that it deserve. Many open source project have no first class support to the said database. Web hosting with Firebird installation is very rare.

Is Firebird not as good as the way I see it? or it just lack some marketing.

Django or Rails for Grails user

I had a wonderful experience with Grails after finishing two projects. If you are a new Java programmer who's looking for a good web frameworks, Grails will be your savior just like what it was to me.

Here's what I like about Grails

• Relatively easy to learn compared to other Java web frameworks.
• Provides the basic things that one need, it's all there and there's not much
  to configure (ex. No XML).
• It's agile!
• Good ORM built on top of Hibernate.
• Grails was developed mostly in Java so it's enterprise-ready and it scales well.
• Uses groovy. Easy to learn Java scripting language
• Although it's not really Java, it does take advantage of the massive amount of Java libraries available, including ones I use like the Java Excel API and JFreeChart.
• I like database stored procedure programming, and calling stored procedure
  
  from Grails is easy as well as executing complex sql queries.
  
• Integrate my choice of Javascript framework easily. I use jQuery.
• Grails helps me code properly because of its MVC pattern and services
  
  concept.
• Grails plug-in are so cool like Quartz and Searchable
  

One of the requirements of my next project is that it be hosted on the internet. My client doesn't care what technology I use but wants the hosting fee to be as low as possible. And as we all know, Java web hosting is not cheap compared to others.

I hope to use a non-Java web framework, Django or Ruby on Rails, that I could be comfortable with because of the knowledge and experience I have with Grails. I'd like to see the things that I like in Grails on them. Right now, I'm gearing towards Django because I know a little of Python.

Comments & Suggestions?

Beginners guide for Google Spreadsheet API

Programming with web API's is a new thing to me, and this is the first web api I have used. I have encounter a number of troubles before getting things to work and that is why I am writing this post as an additional to their guide especially for beginners like me.

I am using python to work with the api, but this could be applicable to other languages since most of the problems I am dealing with are XML related I guess. For this little & simple application that I am trying to create the operations I perform are:

• Connecting to Google Spreadsheet Service
• Opening the spreadsheet and worksheet you wish to work with
• Writing into the spreadsheet
• Reading spreadsheet

Connecting to Google Spreadsheet service is easy, here's the code:

gd_client = gdata.spreadsheet.service.SpreadsheetsService()
gd_client.email = 'your_username'
gd_client.password = 'your_password'
gd_client.ProgrammaticLogin()

Opening the speadsheet and worksheet

As of now, the api doesn't allow us to create a spreadsheet, we can only work on the existing spreadsheet. The first thing to do is to get the key which will tell the api what spreadsheet you want to work with. Currently I am getting the spreadsheet key by opening the document itself and you could get from the url.

You can also code your way to get your spreadsheet's key like this.

spreadsheet_feed = gd_client.GetSpreadsheetsFeed()
for spreadsheet in spreadsheet_feed.entry:
print spreadsheet.id.text.rsplit('/', 1)[1]

Now we need to know which worksheet we want to work with, the only way I know to get the worksheet key is to code it. We select one of the spreadsheet key from the above

spreadsheet_key = 'o04042323673667114780.4164488110474914499'
worksheet_feed = gd_client.GetWorksheetsFeed(spreadsheet_key)
for worksheet in worksheet_feed.entry:
print worksheet.id.text.rsplit('/', 1)[1]

Now we know the spreadsheet key and worksheet key, we can now start reading or writing.

spreadsheet_key = 'o04042323673667114780.4164488110474914499'
worksheet_key = 'od6'
gd_client = gdata.spreadsheet.service.SpreadsheetsService(spreadsheet_key, worksheet_key)
gd_client.email = 'your_username'
gd_client.password = 'your_password'
gd_client.ProgrammaticLogin()

Writing into the spreadsheet

Currently, the API provide 2 ways of writing by row and by cell. There are things you need to do and know when writing by rows. The api treats the first row as the column header, so before you write by rows you need to set the column header manually.

We need first define a dictionary first, before we can write by rows.

dict = {'firstname':'Kervin', 'lastname':'Ang', 'age':29, 'birthday': '9/12/1977'}
gd_client.InsertRow(dict, spreadsheet_key, worksheet_key)

The keys of the dictionary will tell to which column we will write the data. As you can see spaces, capital letters, and special character are disregarded. For example, if your column header are $ Money Market and Php Money Market the keys of dictionary are moneymarket and phpmoneymarket respectively.

This where I spend I lot of making things work, underneath the api deals with a lot of XML/ATOM/RSS as I don't have any idea about it.

To check if the insert row is success, you could do something like this

entry = gd_client.InsertRow(dict, spreadsheet_key, worksheet_key)
if isinstance(entry, gdata.spreadsheet.SpreadsheetsList):
print "Insert row is success."

Writing by cell will be very useful in setting up column header.

Reading spreadsheet

Same with writing into spreadsheet, you could read the spreadsheet by rows or cells. An example of reading all the rows of spreadsheet

list_feed = gd.client.GetListFeed(spreadsheet_key, worksheet_key)
for entry in feed.entry:
print "%s: %s\n" % (entry.title.text, entry.content.text)

You could search or filter rows using the gdata.spreadhseet.service.ListQuery like this

query = gdata.spreadsheet.service.ListQuery()
query.sq = 'age==29'
list_feed = gd_client.GetListFeed(ss_key, ws_key, query = query)

Just like writing into spreadsheet by rows, you need to provide the correct key to match the column head.

With this I was able to retrieve some data from the internet and store them in google spreadsheet for my data analysis, and I am looking for more ways of using it.

Make your source code look cool on your blog with Pygments

After my previous post that mostly contains a python code, I wasn't satisfy at the way it look where I just simply format it with courier font. Luckily I discover Pygments.

From its website, Pygments is a generic syntax highlighter for general use in all kinds of software such as forum systems, wikis or other applications that need to prettify source code. Highlights are:

• a wide range of common languages and markup formats is supported
• special attention is paid to details that increase highlighting quality
• support for new languages and formats are added easily; most languages use a simple regex-based lexing mechanism
• a number of output formats is available, among them HTML, RTF, LaTeX and ANSI sequences
• it is usable as a command-line tool and as a library
• ... and it highlights even Brainf*ck!

Here are the steps I did to prettify my code in the post. (do this also if you don't want to install pygments)

1. Go to http://pygments.org/.
2. Let pygments highlight my code.
3. Once my code are highlighted, I choose other highlighting style to suits my taste.
4. View the page source code. Locate and copy the
   
   
   

   <link rel="stylesheet" href="http://www.pygments.org/media/pygments_style.css">
   
   <div class="hlcode">
   <div class="syntax"><pre>
   ...
   </pre></div>
   </div>
   
   

5. Paste the html code.
6. Put this style link into my template. Using style link could result to unfix highlighting style, instead you can copy the whole css into your html.
   
   
   

   <link rel="stylesheet" href="http://www.pygments.org/media/pygments_style.css">
   

   
   

Hope support for groovy could come soon. Enjoy highlighting your code.

My first Python program excites me: Source code

As someone has requested for the source code, here it is. Improvements are welcome.

#!/usr/bin/env python

from urllib import urlopen

try:
 from xml.etree import ElementTree
except ImportError:

 from elementtree import ElementTree
import gdata.spreadsheet.service
import gdata.service

import atom.service
import gdata.spreadsheet
import atom
import getopt
import sys

import string
from HTMLParser import HTMLParser
import re
from datetime import datetime


class bdoHTMLParser(HTMLParser):
  
   def reset(self):

       HTMLParser.reset(self)
       self.getDate = None

       self.rawdate = None
       self.date = None

       self.key = ''
       self.data = {}

      
  
   def handle_data(self, data):
       if re.search(r'Unit Investment Trust Funds',data):

           self.getDate = data
       elif self.getDate:

           try:
               self.rawdate = data.split(' ', 1)[1].strip()

               self.getDate = None
           except:
               print "Parsing date failed: %s" %(data)

       elif re.search(r'BDO\D+Fund',data):
           self.key = data

       elif self.key:
           self.data[self.key] = data.strip()

           self.key = None

def main():
   # SETUP

   columns_header = ['phpmm', 'phpmmchange', 'mm', 'mmchange',

       'phpbond', 'phpbondchange', 'bond', 'bondchange',
       'balance', 'balancechange', 'equity', 'equitychange',

       'fixedincome', 'fixedincomechange']
   uitf_to_header_map = {'BDO Peso Money Market Fund':columns_header[0], 'BDO $ Money Market Fund':columns_header[2],

       'BDO Peso Bond Fund':columns_header[4], 'BDO $ Bond Fund':columns_header[6],
       'BDO Balanced Fund':columns_header[8], 'BDO Equity Fund':columns_header[10],

       'BDO Fixed-Income Fund':columns_header[12]}
  
   #ss_key = 'o04042323673667114780.4164488110474914499'
   ss_key = 'pHJcUzPY3GTjD6jFA3zvLOQ'

   ws_key = 'od6'
  
   # GETTING THE SOURCE
   page = urlopen('http://www.bdo.com.ph/NAV.asp')

   html = page.read()
   page.close()
   print "Download Data Success"

  
   # PARSE DATA
   parse_data = bdoHTMLParser()
   parse_data.feed(html)

   parse_data.close()
   print "Parsing Success"

   # CONNECT TO GOOGLE SPREADSHEET
   gd_client = gdata.spreadsheet.service.SpreadsheetsService()

   gd_client.email = 'your_username'
   gd_client.password = 'your_password'

   gd_client.ProgrammaticLogin()
   print "Login Success"
  
   # CHECK DATE FORMAT
   if re.match(r'\S{3,9}\s+\d{1,2},\s+\d{4}',parse_data.rawdate):

       temp_date = datetime.strptime(string.join(parse_data.rawdate.split()), '%B %d, %Y')

       parse_data.date = "%s/%s/%s" % (temp_date.month, temp_date.day, temp_date.year)

       print 'Date Format Pass: %s' % (parse_data.date)
      
   else:

       print """Date Format Failed: %s\n
            BDO Log Aborted.\n""" % (parse_data.rawdate)

       return
  
   # CHECK LAST DATE LOG
   query = gdata.spreadsheet.service.ListQuery()

   query.sq = 'date=%s' % (parse_data.date)

   feed = gd_client.GetListFeed(ss_key, ws_key, query = query)

   if len(feed.entry) != 0:
       print """Entry Found.\nBDO Log Aborted.\n

            %s %s""" % (feed.entry[0].title.text, feed.entry[0].content.text)

       return
  
   # CONSTRUCT DATA
   data = {}
   data['date'] = parse_data.date

   for k, v in parse_data.data.items():

       data[uitf_to_header_map[k]] = v
       #print "%s = %s" % (k, v)

   print 'Construct Data Success'
  

   # WRITE DATA
   entry = gd_client.InsertRow(data, ss_key, ws_key)

   if isinstance(entry, gdata.spreadsheet.SpreadsheetsList):
       print """BDO LOG Success\n

            Enter: %s""" % (data)
  
  
if __name__ == '__main__':

   main()
  
  
  

My Grails Practices

After learning Grails and finish 2 small project with it in 4 months time, here are a number of the practices I do to develop faster and comfortable for me. As I am new to Grails as well as to java web development, the practices here may not be wrong and your comments are very much welcome.

A controller's action performing post and get.

I have a lot of that in my project, where it really helps me organize my code to be more readable and maintainable and less confusing. This is not against to be CRUD pattern that grails generate, but somehow I feel not comfortable with it, because I have to think two action's name instead of one. For example, I have view employee profile page where you can extends the the employee's contract or terminate the employee's contract. Here's how I code it

def extendEmployeeContract {
if (request.method == 'GET') {
// preparing the page for extending the employee's contract
}
else if (request.method == 'POST') {
// performing the employee's contract extension
}
}

instead of

def createEmployeeContractExtension {
// preparing the page for extending the employee's contract
}

def saveEmployeeContractEntersion {
// performing the employee's contract extension
}


Restricting an action to post only if applicable

Here's how grails make it easy for you, just define it in the allowedMethods attributes.

def allowedMethods = [checkEmployee:'POST']

This makes your application much more secure.

Do you manual assignment after object.properties = params

Sometimes I need to do some manual assignment when an attribute is your domain class, and be able to validate the object that we want to assign to an object's attribute. Doing so we avoid a hibernate error saying that a object's identifier is alter from 1 to 2 which are the id of the object.

Use java.sql.date for date attribute

Using java.sql.date could result into a date field on your database while java.util.date could give you a datetime field in your database which give me a problem when I perform a query with the dynamic find method. This gives me additional work to do because the object.properties = params won't work here but this bug is fix and will be on the next release.

Another reason I use java.sql.date is because of how I implement my user interface. I am not using the gsp datepicker tag, instead I use a regular text field which is much faster to input in for the user and you can use any javascript datepicker solution. For the date format I use yyyy-M-d and it suit very well to java.sql.date, use the java.sql.date.valueOf() static method. Here's how I implement it.

employee.birthday = params.birthday =~ /\d{4}-\d{1,2}-\d{1,2}/ ? java.sql.date.valueOf(params.birthday):null

This validates the user's input and if it's a invalid format, a null will assign to the date attribute which trigger the null constraints I define.

More will be from you, I be glad to hear it.