Satya's blog - 2007/09/

This weekend I've been trying to write a train game. Graphical, top-down simulation of railway tracks and yards. And I'm doing it in Python using PyGTk and PyGame.

Why that language and those GUI (graphical user interface) frameworks? Because over 2 years ago, some people convinced me that if I was going to make a graphical game, NOT in a web browser, then pygame was the thing to use. Pygame is obviously Python. So I'm having to learn Python, while trying to write a graphical native-OS (not browser) game. My forté is web programming, where the UI is built for you and the UI programming language (HTML) is dead simple. This is a big change.

So, 2 years ago, I built a few Python classes that could throw up an SDL window, draw a single track, and let me run a single train along it. Nothing dynamic, track and train hard-coded, only able to control direction and speed. It didn't even have acceleration. At that time I had started with the game's core.

This weekend I decided to start on the user interface. I needed something that would work across platforms (which does not mean "Oh we have both kinds, XP *and* 2000!") and wouldn't be too hard.

I considered switching to Java, another language that's new to me. The advantage would be, I could be forced to learn Java. That was also the disadvantage. I considered Javascript, the advantage being I am already deeply familiar with web programming. The disadvantage is that Javascript has very little drawing capability; some libraries are available that apparently simulate lines with div tags. Horrible. It works, but still! Besides, having hundreds of lines (track segments) would probably be slow and memory intensive, since each tiny div drags along its entire DOM object definition with it.

So, back to Python. It's a good language to learn, I've been told. And I've already got the afore-mentioned code samples. Okay, so I need to generate a user interface. Enter PyGtk (however it's capitalised). After a few false starts, I have a File->Open menu and now I can create a pygame drawing surface and I'm ready to show a map.

Then I find Rail World and Freight Yard Manager (FYM), which completely take away my motivation for doing this. They seem so slick, and better. And compatible with Yard Duty, the now-defunct but still being run original top-down freight rail simulation. Lots of bugs, no longer maintained, but still in use. And has hundreds of maps. Whose lack of Linux-compatibility and un-maintenance inspired me to start my project in the first place.

Then I find out that Rail World won't work with my Java for some reason, and FYM just plain won't run even with what I think is an appropriate .NET version (and won't run in wine, the windows emulator on Linux). Oh, and FYM is Windows-only. So, I might do my game after all :-) It does need to be compatible with Yard Duty, and that sucks. Maybe a converter, but first let me get *something* working.

Update: Rails World does work -- with Sun Java 6, which you have to enable on Ubuntu thus:
install the Sun Java 6 packages (sun-java6-*), edit /etc/jvm to include the line:
/usr/lib/jvm/java-6-sun
and run:
update-alternatives --config java
(as root)

Last updated: Oct 01 2007 15:46

Let's say you're like me and want your production log to be in /var/log/rails/yourapp.log, so you can keep your rails logs together for rotation etc. You still want to log your development Ruby on Rails log into log/development.log, as usual.

In config/environments.rb put this line:
config.logger = Logger.new("/var/log/rails/yourapp.log")
That's all.

UPDATE: The better way is to set
config.log_path='/var/log/rails/yourapp.log'
in config/environments/production.rb instead of config.logger as above.

You could have placed that line in config/environment.rb within the Rails::Initializer.run do |config| ... end block. That would have changed logging for all environments; development, production, and test.

Oh, you want to rotate the logs? (That's when you set up the logrotate package to periodically move your current log to yourapp.log.1 and truncate it, so eventually you end up with numbered or dated log files instead of one huge log file going back several months and occupying 200MB.) Easy enough, drop in a file /etc/logrotate.d/rails which contains:

weekly
missingok
rotate 52
compress
delaycompress
notifempty
sharedscripts
copytruncate
/var/log/rails/*.log {
    postrotate
        /etc/init.d/mongrel restart yourapp1
        /etc/init.d/mongrel restart yourapp2
    endscript
}

You don't need the postrotate..endscript block unless you want to restart mongrel after the log rotates. Since this script will cause weekly rotation, it's probably a good idea to restart mongrel as well. If you want to restart all the mongrel apps, you can of course leave off "yourapp1" etc, and just do:

    postrotate
        /etc/init.d/mongrel restart
    endscript

Last updated: Sep 24 2007 14:00

This HOWTO explains how, given your database, a PDF form, Ruby on Rails, plus a few other things, you can produce filled-out PDF documents. This assumes Debian/Ubuntu.

You will need a PNG to PNM convertor, an XSL transformer, Inkscape for producing and converting SVG files, and a PNG quantiser (optional).

Overview: The page layout template is produced as an SVG file in Inkscape, from an existing (blank) PDF. Inkscape is also used by Ruby on Rails (RoR) to convert the merged SVG into a PDF that is served to the user via the web.

First, get all the stuff besides Ruby on Rails and so on, stuff which isn't likely to be on your average Rails server:
apt-get install netpbm libxslt-ruby inkscape pngquant
(netpbm for pngtopnm)

Making the template

Convert the original PDF to PNM
pdf2ps | pstopnm or whatever

Open the PNM file in inkscape. This gets the right size of page.
Delete everything.
Create new layers: scan, trace, borders, boilerplate, dynamic.
Set fonts to something ghostscript understands, like the URW family.

Import the PNM into the 'scan' layer. That is, keep the scan layer selected in the layers dialog, and then Import.

Select all, Path -> trace bitmap (long, memory intensive process starts)
Move the path to the 'trace' layer.
Delete the extra nodes, i.e. text etc. (long, memory-intensive process ends)

Put any extra "drawn" stuff you need into the 'borders' layer, such as empty checkboxes or borders that don't show up properly in 'trace'

In 'boilerplate', put all the text that won't change, i.e. what's always there.

In 'dynamic' layer, put placeholders for the fields. I'd insert as text the field name from the database, such as where the last name would normally go I'd put the actual string 'last_name'. This will later be replaced by XSL markup.

Save as SVG.

The SVG file is just XML. Fix the saved SVG file as an XSL stylesheet by adding these lines in a text editor:
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:template match="/claim">
instead of claim, whatever the xml root node is, and close the tags at the bottom of the file.

Use xsl tags instead of the dynamic fields elements. The text 'last_name' becomes: <xsl:select name="last_name" /> or whatever the correct XPath is.

Transforming the template in RoR

Now you write the RoR methods to convert your data to PDF. Take the data from the database, convert it to XML, and "transform" the XML using the XSL stylesheet you created in the previous section. Then you will use Inkscape to convert the transformed XML into a (PNG, then a) PDF.

Get your XML and transform it with (Ruby). The following can probably be a method of your ActiveRecord model, say, to_pdf:

xml_string=to_xml
require 'xml/libxslt'
xslt = XML::XSLT.new()
xslt.xml = "string containing the xml"
xslt.xsl = "your.xsl"
xslt.save("/tmp/something.svg")

to_xml is another method that works something like this:

def to_xml
    buffer=""
    xm=Builder::XmlMarkup.new(buffer)
    xm.instruct!
    buffer += xm.report {
       xm.employee_name("#{employee_name}")
       %w(address1 address2 name).each do |col|
           xm.tag!('employer_'+col, employer.send(col))
       end
       #and so on
    }
    return buffer
end 

Making the PDF

Back to your to_pdf method. Remember, stuff like #{id} in a string gets the current object's id:

fileprefix='/tmp/your_app_#{id}"
inkscape="/usr/bin/inkscape #{fileprefix}.svg --export-background=#ffffff --export-png=#{fileprefix}.png"
system(inkscape)
system("pngquant 8 #{fileprefix}.png")
system("pngtopnm #{fileprefix}-fs8.png | pnmtops | ps2pdf - > #{fileprefix}.pdf")

Basically just a series of system() calls.

Then read the PDF in like so:
pdf_data = IO.read("#{fileprefix}.pdf")

You should save 'data' into the database. It's raw PDF code, so use a blob or other large binary type. When you create this column in the migration, use something like: add_column :table, :pdf_data, :binary, :limit => 512.kilobyte

If your PDF table is separate from your data table (may make things easier on the db engine), you probably want something like this:

if pdf_id.nil?
    pdfo=Pdf.new
else
    pdfo=Pdf.find(pdf_id)
end
pdfo.data = IO.read("#{fileprefix}.pdf")
pdfo.save
update_attributes(:pdf_id => pdfo.id)

where your table belongs_to :pdf and the pdfs table has_one :your_data_table

I also use a 'dirty' column, so anytime the controller's pdf method is hit, it checks the dirty bit; if set, it calls to_pdf on the object. Then it sends_data. Pseudo-code:

def add
    obj.dirty=true
    obj.save
end

def pdf
    obj=Model.find(id)
    obj.to_pdf if obj.dirty
    send_data(obj.pdf.data, :type => 'application/x-pdf', 
    :filename => "something_#{obj.id}.pdf")
end

Inkscape uses Gnome's VFS, which wants the user's home directory to be writeable. The user it runs as, that is. Now your RoR is probably running as a mongrel process, as www-data or something. So, create a user for running this app, call it inkscape-pdf-makr or something. Run mongrel as that user by putting USER=inkscape-pdf-makr in /etc/mongrel/sites-enabled/your-site.conf. Make sure the permissions on log/ and tmp/ are correct.

A couple of months ago I acquired a vintage 1987 IBM Model M keyboard. A genuine 1401. This is geek history. I cleaned it up and, due to whining in a certain newsgroup, got a new cable for it. It is now my work keyboard. Pictures are now posted in my albums.

Why I don't watch (some) TV series: Because they drag on and on, never reaching a resolution. At least Quantum Leap told us that Sam never got home. At least TNG didn't *have* an ultimate goal. At least Ross and Rachel-- no, that doesn't count, it's a sitcom. I'm talking about shows like Heroes (they trying to save the planet or something? they're never going to do it), Lost (do they ever get off the island?), the X-Files (do we ever resolve the conspiracies, or is it just layers and layers of fluff?), etc. I saw some of these, others I avoid because I've been bitten before (season 7, now come on, resolve *something* as a non-specific example). I'm afraid they will jump the shark.

Almost forgot: Battlestar Galactica! (Must be read in a Shirish Kanekar style, the same way he says "Shakti Kapoor! To tar akhil bharatiya balatkar sanghatnecha adhyaksha ahey!" -- He's referring to the old Shakti Kapoor, the one that was. The one that came before the sane police officer of 'Khiladi' and before the complete idiot of 'Coolie No. 1'). I do watch it because it's a great show, great story, but are they ever going to find Earth? Or get destroyed? See, there's no closure there.