Heroku Tip; Use a Ping or Uptime Service As a Cron Job.

This is a pass-along tip from Toby Hede (http://stackoverflow.com/users/14971/toby-hede) on StackOverflow. Just give the Ping service the URL of the controller doing the work instead of your root page.

Rails Tip - How Do I See Params From My Forms?

I like to document what my forms are sending to my action methods. So most of the time, I create a method in my controller like this:

def to_export
  render :text => params.inspect
end

Then I use copy and paste to put the result into my code as a comment.

# {
#   "commit"=>"Upload File", 
#   "authenticity_token"=>"17FvBH+Z4Meni6WvsCdPByrrk751BV9mNklzKDAo2Vc=", 
#   "action"=>"do_import", 
#   "import_criteria"=>{
#     "primary_language_name"=>"Creole", 
#     "secondary_language_name"=>"English"
#   }, 
#   "form"=>{
#     "file"=>"traveling_haiti.txt"
#   }, 
#   "controller"=>"import"
# }

Another Fix for EMAIL not authorized to access APP

I wanted to change the primary email associated with my Heroku account. So I removed the email from the authorization list. The next time I pulled from git, I got the following error:

heroku@david-medinets.otherinbox.com not authorized to access crisiscamp-translation

The email address is pulled from your public/private keys which can be recreated using the following command:

ssh-keygen -t rsa -C EMAIL

Using Formtasic Without ActiveRecord

Let's start with a model:

class PrimaryLanguage
  attr_accessor :name
  def initialize(name)
    @name = name
  end
end

Then create an object in a controller:

class AbcController < ActionController::Base
  def aaa
    @primary_language = PrimaryLanguage.new('English')
  end
end

Design a form:

<% semantic_form_for @primary_language, :url => select_primary_language_path, :html => { :id => 'primary_language'} do |form| %>
  <%= form.input :name, :as => :select, :collection => @languages, :required => false, :label => "Primary Language:<br/>", :include_blank => false %>
<% end %>

And finally handle the user's input;

class XyzController < ActionController::Base
  def bbb
    primary_language_name = params[:primary_language][:name]
    # do something
  end
end

When Coloring Hover Always Using Anchor CSS.

I just found out that you need to specify both a and a:hover in order to color text when the mouse hovers over a link. Using the a:hover does not work in isolation.

Plug A Security Hole in RubyGems By Mailing Ruby Install Dir ReadOnly!

When creating a Ruby gem, developers can specify a list of executable files. These executives are copied into the same directory where the Ruby binary is located. While very convenient, it opens a huge security hole.

It would be fairly easy for someone to provide an alternate version of the Ruby executable.

If possible, write-protect your ruby install directory to prevent this avenue of attack. If you run Windows and are paranoid, backup the Ruby install directory so you have something for future comparison.

How to Resolve Heroku's 'not authorized to access newname' Git error.

When you see the 'not authorized to access newname' error, it probably. means that your Git configuration is incorrect. For example, my .git/config file had the following:

[core]
 repositoryformatversion = 0
 filemode = true
 bare = false
 logallrefupdates = true
[remote "heroku"]
 url = git@heroku.com:newname.git
 fetch = +refs/heads/*:refs/remotes/heroku/*

Notice the highlighted words. Just change newname to the actual project name and you should be fixed.

Scripting the Heroku Push

This tip falls under the KISS principle. I recently started using the Heroku service to run some Rails websites. After making changes and committing them to your local Git repository, those changes are pushed to Heroku using the command git push heroku master. Since I'm likely to forget and since it's four words I created a file called script/push with that command. Here are the steps.

1. Create a file called script/push with one line: git push heroku master.
2. Run chmod +x script/push
3. Run git add script/push.
4. Run git commit -m "scripting the Heroku push".
5. Run script/push.

That's it. Just script/push whenever you need to deploy.

Most Efficient Food in Cafe World

2009-Nov-23 Updated with new foods. Changed net income to reflect the 15 coin charge for cleaning the stove.

2009-Oct-31 Updated with crab bisque, halibut and peking duck.

My kids like this game. I needed a way to make it more interesting. Here is the list of food they can make, in order of efficiency.

food                     cost   income  net  hours   minute secosnds  net/seconds
bacon cheeseburger         15   52      22              5       300        0.07
overstuffed peppers      1300 4300    2985    12              43200        0.07
kung pao stir fry         600 1600     985     4              14400        0.07
king crab bisque         1300 6685    5370    24              86400        0.06
chips and guacamole        10   36      11              3       180        0.06
super chunk fruit salad    35  100      50             15       900        0.06
atomic buffalo wings      350  960     595     3              10800        0.06
buttermilk pancakes       250  400     135             45      2700        0.05
tony's classic pizza      400 1300     885     5              18000        0.05
chicken gyro and fries     45   88      28             10       600        0.05                               
voodoo chicken salad      700 2675    1960    12              43200        0.05
herbed halibut            700 4500    3785    24              86400        0.04
crackling peking duck     900 3600    2685    18              64800        0.04
jumbo shrimp cocktail      65  148      68             30      1800        0.04
savory stuffed turkey     700 3600    2885    22              79200        0.04
tikka masala kabobs       215  360     130     1               3600        0.04
spagetti and meatballs    450 1375     910     8              28800        0.03
spitfire roasted chicken  600 3200    2585    24              86400        0.03
french onion soup         175  615     425     4              14400        0.03
triple berry cheesecake   400 1650    1235    12              43200        0.03
caramel apples             90  300     195     2               7200        0.03
homestyle pot roast      1800 5750    3935    48             172800        0.02
staked steak              300 2010    1695    24              86400        0.02
pumpkin pie               200 1060     845    12              43200        0.02

Forest Options of ps Command

Some computer system that I use have a ptree (or process tree) command. It is very nice to see processes and their children. In fact, I am disappointed when working on systems without ptree. And then I learned about the forrest option of the ps command.

ps faux

Wonderful. Makes me happy. RTFM!

United States' Official Repository of Foreign Place-names (more entity extraction fun)

https://www1.nga.mil/ProductsServices/GeographicNames/Pages/default.aspx - This is a fantastic resource if you need a list of place names. You can download a file containing names covering the whole world.

Names in Non-English Languages (continuing a theme)

The title is horrible but clear, at least to me. I only have one link so far, but it's a good one.

• http://blog.jclark.com/2007/12/thai-personal-names.html - There's an election coming up in Thailand on December 23rd and the streets are lined with election posters. As a bit of an i18n geek, I find it interesting that the posters almost all make the candidates' first names at least twice as big as their last names. If you're also an i18n geek, your reaction might well be: "it must be because Thais write their family name first, followed by their given name". But you would be wrong. Thais have a given name and a family name; the given name is written first, and the family name last.

Entity Extraction Links

• Software
• Ruby
• http://github.com/hypomodern/flex-attributes/ - Originally by Eric Anderson, some of that code still remains. If you’re not into this version, check his out at rubyforge.org/projects/flex-attributes/ See Hypomodern::FlexAttributes for usage information.
• http://www.opencalais.com/ - Calais is a rapidly growing toolkit of capabilities that allow you to readily incorporate state-of-the-art semantic functionality within your blog, content management system, website or application.
• http://www.aktors.org/technologies/annie/ - Open Source Information Extraction from The University of Sheffield; ANNIE is an open-source, robust Information Extraction (IE) system which relies on finite state algorithms. ANNIE consists of the following main language processing tools: tokeniser, sentence splitter, POS tagger, named entity recogniser.
• http://www.searchenginecaffe.com/2007/03/java-open-source-text-mining-and.html - Jeff Dalton's List of Java Open Source NLP and Text Mining tools
• http://gate.ac.uk/ - GATE as an architecture suggests that the elements of software systems that process natural language can usefully be broken down into various types of component, known as resources
• http://www.casos.cs.cmu.edu/projects/automap/ - AutoMap is a text mining tool that enables the extraction of network data from texts. AutoMap can extract three types of information: content analytic (words and frequencies), semantic networks, and meta-networks.
• http://www.sra.com/netowl/ - SRA developed NetOwl®, a suite of rich text mining tools, to discover and extract the knowledge found in free-form text documents and turn it into actionable information. NetOwl has been refined over more than a decade of research and development. Our team of researchers and engineers continue to expand NetOwl’s capabilities to keep pace with evolving information needs.
• http://www.inxightfedsys.com/products/sdks/tf/default.asp - Out of the box, Inxight ThingFinder automatically identifies and extracts more than 35 key entities - such as people, dates, places, companies or other things - from any text data source, in multiple languages. This ability to automatically identify and classify relevant entities makes ThingFinder one of the most powerful text analysis and extraction tools on the market. Using Inxight ThingFinder, developers can maximize and extend the value of their applications by enabling end-users to quickly find the most important pieces of information within large volumes of documents.
• http://incubator.apache.org/uima/ - UIMA enables applications to be decomposed into components, for example "language identification" => "language specific segmentation" => "sentence boundary detection" => "entity detection (person/place names etc.)". Each component implements interfaces defined by the framework and provides self-describing metadata via XML descriptor files. The framework manages these components and the data flow between them. Components are written in Java or C++; the data that flows between components is designed for efficient mapping between these languages.

Articles

• http://en.wikipedia.org/wiki/Named_entity_recognition - Named entity recognition (NER) (also known as entity identification and entity extraction) is a subtask of information extraction that seeks to locate and classify atomic elements in text into predefined categories such as the names of persons, organizations, locations, expressions of times, quantities, monetary values, percentages, etc.
• http://www.semanticuniverse.com/articles-entity-extraction-and-semantic-web.html - Entity Extraction is the process of automatically extracting document metadata from unstructured text documents. Extracting key entities such as person names, locations, dates, specialized terms and product terminology from free-form text can empower organizations to not only improve keyword search but also open the door to semantic search, faceted search and document repurposing. This article defines the field of entity extraction, shows some of the technical challenges involved, and shows how RDF can be used to store document annotations. It then shows how new tools such as Apache UIMA are poised to make entity extraction much more cost effective to an organization.
• http://broadcast.oreilly.com/2009/02/how-entity-extraction-is-fueli.html - How Entity Extraction is Fueling the Semantic Web Fire; short commentary on Apache UIMA and a few other tools.

Installing Ruby and RubyGems On Ubuntu 8.10

     # ruby
    cd ~/rn
    wget ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-p0.tar.gz
    tar -xzvf ruby-1.9.1-p0.tar.gz
    cd ~/rn/ruby-1.9.1-p0
    ./configure
    make
    make test
    sudo make install
    # zlib
    cd ~/rn/ruby-1.9.1-p0/ext/zlib
    sudo apt-get install zlib1g-dev
    ruby extconf.rb --with-zlib-include=/usr/include --with-zlib-lib=/usr/lib
    make
    sudo make install
    # gem
    cd ~/rn
    wget http://rubyforge.org/frs/download.php/45905/rubygems-1.3.1.tgz
    tar -xzvf rubygems-1.3.1.tgz
    cd rubygems-1.3.1
    sudo ruby setup.rb
    # fix ruby executable
    cd ~/rn/ruby-1.9.1-p0
    sudo make install

Using Rails Gem to Generate a New Project on Ubuntu

Using the rails installed by apt-get was a bit confusing because the rails project being generated was not the rails version that I expected. I had installed the rails gem v2.2.2 but the config/environment.rb file showed rails v2.1.

I fixed this issue by removing the default rails.

sudo apt-get remove rails

And then using an alias:

export GEMDIR=`gem environment gemdir`
export RAILS_VERSION=`gem list rails | grep "rails " | cut -b8- | awk '{sub(/\)/, "");print}'`
alias rails='ruby $GEMDIR/gems/rails-$RAILS_VERSION/bin/rails'

Now when I do

rails dool

The project uses the right version of rails!

The --include-activation and --aasm parameters of the Restful Authentication Rails Plugin are mutually exclusive

I was following the installation instructions for the Restful Authentication Rails plugin. So I executed:

script/generate authenticated user sessions --include-activation -—aasm --rspec

This resulted in:

premature end of regular expression: /\A

After digging into the source code, I found that the parse! method in /usr/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/rails_generator/options.rb swallowed exceptions. Here is the original code.

def parse!(args, runtime_options = {})
  self.options = {}
  @option_parser = OptionParser.new do |opt|
    opt.banner = banner
    add_options!(opt)
    add_general_options!(opt)
    opt.parse!(args)
  end

  return args

  ensure
    self.options = full_options(runtime_options)
end

The OptionParser initialization may cause exceptions which are not properly handled. I don't know what should be down but added the following code immediately before the ensure line at least will show the exception.

    rescue => e
    puts("I caught a #{e.class.to_s} with message #{e.to_s}")

Now that the exception is displayed, I saw one of my underlying problems:

I caught a OptionParser::InvalidOption with message invalid option: --include-activation

So I learned that when the --aasm parameter is used, then the --include-activation must not be specified.

Using the Visualize Models plugin

This is a good plugin. It works with GraphViz to generate diagrams of your object model. After installation, I ran into a problem that Visualize::Inflector was not found. This was easily resolved by using ActiveSupport::Inflector in visualize_models.rb.

The Myth of the Hero Programmer

Fortunately, I have met few self-appointed heroic programmers over the years. However, late last year I was saddened to met another. People on truly agile teams might never have met this dying breed of programmer so let me present the archetype by borrowing from the Highlights magazine:

Goofus: Writes complex hard-to-understand code to prove superiority.
Gallant: Writes clear easy-to-understand code to enhance team dynamics.

Goofus: Writes code with side-effects to show his mastery of arcana.
Gallant: Writes simple code that is easy to maintain and debug.

Goofus: Disdains and deletes comments.
Gallant: Describes code context to aid maintenance and provide road maps for new team members.

Goofus: Speaks in imperative words and derisive tones.
Gallant: Speaks with humility and guides people to solutions.

The Ruby community should guard against any tendency towards heroic programmers to avoid some of the complaints made against the Perl community. Too many 'perlers' loved complexity for its own sake which lead to unmaintainable code and a bad reputation for the language in general.

Let's focus on the agile concepts of teamwork, shared code ownership, and collaboration. They are much healthier for the community.

What Goofus/Gallant experiences have you had?

Avoiding the undefined method `use_transactional_fixtures=' error.

After adding rspec to my project and generating a model with rspec_model, I tried to run the generated spec test:

$ spec -cfprofile spec/
.../toshi/spec/spec_helper.rb:12: undefined method `use_transactional_fixtures=' for # (NoMethodError)
 from /usr/lib/ruby/gems/1.8/gems/rspec-1.1.11/lib/spec/runner.rb:184:in `configure'
 from /home/medined/Workspaces/toshi/spec/spec_helper.rb:8
 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
 from /home/medined/Workspaces/toshi/spec/models/book_spec.rb:1

After a fruitless internet search, I tried using a rake task:

$ rake spec
.
Finished in 0.061996 seconds
1 example, 0 failures

Adding RSPEC to a Rails Project

$ rails toshi
$ script/generate rspec
Couldn't find 'rspec' generator
$ cd toshi/vendor/plugins
$ git clone git://github.com/dchelimsky/rspec.git
$ git clone git://github.com/dchelimsky/rspec-rails.git
$ cd ../../
$ script/generate rspec
      create  lib/tasks/rspec.rake
      create  script/autospec
      create  script/spec
      create  script/spec_server
      create  spec
      create  spec/rcov.opts
      create  spec/spec.opts
      create  spec/spec_helper.rb

How is a Use Case Different From a User Story?

http://alistair.cockburn.us/Elephant+carpaccio - Alistair provides a way to think about Use Cases and User Stories. In essence, he says that Use Cases provide the overall shape of the application while User Stories provide the details.

How do I restore and change gedit key assignments?

Use the vi editor to modify the .gnome2/accelsgedit. Editing the file with gedit does not work; the changes are not saved. I had reassigned my ^s key away from File>Save. I order to get the key assignment back, I re-commented the following line:

(gtk_accel_path "<Actions>/GeditWindowActions/FileSave" "<Control>s")

RCOV - Covering unit, integration, and functional testing in one analysis.

Add the following code into a file called lib/tasks/rcov.rake. Notice that I used the find to gather the relevant tests then the xargs to form them into a nice line which can be added to the rcov command.

namespace :test do

  desc 'Tracks test coverage with rcov'
  task :coverage => :environment do
    rm_f "coverage"
    rm_f "coverage.data"

    rcov = "rcov --sort coverage --rails --aggregate coverage.data --text-summary -Ilib -Itest -T -x gem/*,rcov*"

    files = `find . -name *_test.rb | xargs`

    rcov = "#{rcov} " + files
    puts rcov
    system rcov

    system("firefox coverage/index.html")
  end
  
end

The new rake command can be executed using rake test:coverage

Advanced Rails Recipes - Chapter Two - The missing copy method.

I am reading through the Advanced Rails Recipes book. In chapter two, it refers to an Event.copy method which is not shown in the book. Shown below is the missing method from app/models/event.rb.

class Event < ActiveRecord::Base

  def self.copy(other)
    self.new(other.attributes.merge(:name => "Copy of #{other.name}"))
  end

end

JBoss Drools CLASSPATH Error; The Eclipse JDT Core jar is not in the classpath

JBoss Rules v4.0.7 is compatible with Eclipse v3.3 or higher. However there is a simple workaround. Run the following commands:

export ECLIPSE_PLUGINS=/home/medined/Desktop/eclipse-3.4/plugins
ln -s 
  $ECLIPSE_PLUGINS/org.eclipse.jdt.core_3.4.0.v_874.jar 
  $ECLIPSE_PLUGINS/org.drools.eclipse_4.0.7/lib

Getting "Client.InvalidKeyPair.NotFound" Error With Selenium Grid

When running the cap grid:boot command, I received the following error:

Client.InvalidKeyPair.NotFound: The key pair '/home/medined/.ec2/affy.pem' does not exist

It turned out that the EC2_KEYPAIR_NAME system variable was set incorrectly. You can find the correct value using the ec2-describe-keypairs command. Here is an example of its use:

$ ec2-describe-keypairs
KEYPAIR affy c3:e7:5f:09:50:66:f5:2b:3d:2b:ac:1d:09:52:9d:34:29:85:1b:76

So the correct value for me is affy, not the filename.

Using Capistrano to Bundle Amazon EC2 Images

I've found the following bit of Capistrano script to be useful when tinkering with EC2 images. I can make a change and then bundle the image with just one command: cap admin:bundle

The most difficult part of this script was finding out how to upload files to the remote instance using scp. The rest is fairly straightforward. I am only posting this script because I haven't seen this functionality posted anywhere else.

# capfile for Amazon EC2 Bundling. Normally, a yaml file would be
# hold configuration parameters. I am using environment 
# variables because they are also used by the EC2
# programs - why have the information in two places?

# This script assumes that you are only bundling one
# instance at a time.

# EC2 Instance Hostname to be bundled.
ec2_hostname = ENV['BUNDLE_EC2_HOSTNAME']

# S3 Bucket Name (where the bundle is uploaded)
s3_bucket_name = ENV['BUNDLE_S3_BUCKET_NAME']

# If you are using ElasticFox (and if not, why not?) that
# you created in the KeyPairs tab. Specify the full
# path to the file.
ssh_keypair_file = ENV['BUNDLE_SSH_PRIVATE_KEY_FILESPEC']

# Amazon Web Services Info
amazon_account_id    = ENV['AMAZON_ACCOUNT_ID']
amazon_access_key_id   = ENV['AMAZON_ACCESS_KEY_ID']
amazon_secret_access_key = ENV['AMAZON_SECRET_ACCESS_KEY']

# Amazon EC2 Info
cert_filespec     = ENV['EC2_CERT']
private_key_filespec   = ENV['EC2_PRIVATE_KEY']

# We are only dealing with one EC2 instance.
role :libs, "#{ec2_hostname}"

# I like working with ElasticFox which seems to only
# support using the root user.
set :user, "root"

ssh_options[:keys] = ssh_keypair_file

namespace :admin do

 task :bundle do

  # Copy the certificate and private key to the remote computer.
  upload("#{private_key_filespec}", "/mnt", :via => :scp) 
  upload("#{cert_filespec}", "/mnt", :via => :scp)

  # Extract just the filename from the filespec.
  private_key_filename_start = private_key_filespec.rindex('/') + 1
  private_key_filename_end = private_key_filespec.length
  private_key_filename = private_key_filespec[private_key_filename_start, private_key_filename_end]

  # Extract just the filename from the filespec.
  cert_filename_start = cert_filespec.rindex('/') + 1
  cert_filename_end = cert_filespec.length
  cert_filename = cert_filespec[cert_filename_start, cert_filename_end]

  # Remove any old image.
  run "rm --force /mnt/image /mnt/image.*"

  # Create the EC2 Bundle
  #run "ec2-bundle-vol -d /mnt -c /mnt/#{cert_filename} -k /mnt/#{private_key_filename} -u #{amazon_account_id} -r i386"

  # Upload the EC2 Bundle to S3
  run "ec2-upload-bundle -b #{s3_bucket_name} -m /mnt/image.manifest.xml -a #{amazon_access_key_id} -s #{amazon_secret_access_key}"

  exec "ec2-register #{s3_bucket_name}/image.manifest.xml"

 end

 task :update do
  run "apt-get update"
  run "apt-get upgrade -y"
 end

end

Using SCP with Capistrano

It took me several hours to find out how to send a file to my remote computer via scp using Capistrano. It's actually quite easy. There is a method called upload which has a via option. Here is an example of its use:

upload("products.txt", "/home/medined", :via => :scp)

If you need to change the files name, you can also specify it in the second parameter. For example,

upload("products.txt", "/home/medined/products_new.txt", :via => :scp)

Capistrano, ElasticFox & EC2

What a nice combination of tools! I just installed Capistrano and created a capfile. From then on, I could run commands across all my virtual istances in one shot. This setup took just minutes. The only change that I made to my existing instances was to add a capistrano userid. My capfile looks like this:

role :files, 
  "capistrano@ec2-75-101-225-105.compute-1.amazonaws.com", 
  "capistrano@ec2-67-202-59-118.compute-1.amazonaws.com"

namespace :files do
  desc "Show free disk space"
  task :show_free_space, :roles => :files do
    run "df -h /"
  end
end

The best thing of all? No XML configuration!

ElasticFox: How to SSH to Amazon EC2 Image Without Entering a Password

1. Download and install the ElasticFox plugin.
2. Enter your credentials.
3. Enter your account information.
4. Select the KeyPairs tab.
5. Click Create a New Pair.
6. Save to a file; for example ~/.ec2/affy.pem.
7. Select the AMIs and Instances tab.
8. Click Launch Instance(s).
9. In the dialog box, select the keypair that you created in step 5.
10. When the AMI is running, you should be able to connect to it without a password prompt.

Captain Kirk Teaches SCRUM and Agile Development

Episode One: Change the Playing Field

"The odds are against us and the situation is grim." - Star Trek: Generations

STARDATE 11234.2

Students, Starfleet has asked to me say a few words about SCRUM while I'm visiting your base. Fortunately, my meeting with the Ambassadors isn't until 0930 so I have just enough time to talk and then answer a few questions.

Some years ago, I was in a situation that by most accounts (especially Bones') would be considered bleak. Our software team was handed a Klingon application to calculate the expansion rate of exploding warheads. Our Quality Assurance staff was located in the beta quadrant. While my Klingon interpreter and product owner (Kveld of house K'mpok) was in the brig for attempting to kill Scottie. I needed to act, and fast! But what were my options?

Yes, I could have outsourced the work to the Romulons. But they are the enemy! Let's get serious. Other suggestions?

Ok, that's another approach. We could bring everyone together onto a single starbase to form a co-located team. That suggestion is a classic. And workable except for the inter-species issues. My product owner, the klingon in the brig, is also responsible for other projects. He can't be co-located for all of them, can he? Additionally, co-location means additional costs in the form of housing, food, time away from family. In this situation, co-location is not possible.

Hmm. Good idea. Use the holodeck as a virtual meeting room. I like that and that is the approach that we took. Along with some others as well.

Since the team is so dispersed the issue of time was quite important. The team agreed to a daily standup (a scrum, some like to say) but at what time? The meeting time changed every three months. Thus, no one location was impacted more than another - all were inconvenienced equally.

Scottie agreed to import a case of Ra'taj for Kveld and to refrain from further mentioning his mother's similarity to a Gagh.

The approach that I take to every situation is:

What is the operant set of relationships and rules? And how can they be changed to resolve the issue?

Once you know how the forces arrayed against you are deployed, you can find the levers needed to tip the scale in your direction.

Any questions?


Thank you.

Dup dor a'az Mubster (live long and prosper)

Getting flickrfs to Work on Ubuntu Heron

I found this site which provides step-by-step instructions how to use flickrfs in order to mount your Flickr account as a file system. The steps required a little tweaking to work. Here are the steps that I followed:

1. sudo apt-get install libfuse2 fuse-utils python2.4-fuse imagemagick - the original instructions used a capital i for imagemagick.
2. sudo modprobe fuse
3. sudo chmod 755 /bin/fusermount - the original instructions used /usr/bin
4. sudo chmod u+s /bin/fusermount - the original instructions used /usr/bin
5. sudo chmod 666 /dev/fuse
6. cd ~
7. Download flickrfs package from sourceforge.
8. tar -xzvf flickrfs-*.tar.gz
9. screen -S flickrfs
10. lsmod | grep -i fuse # Rerun commands in step 2 if not loaded.
11. cd flickrfs-1.3.95 - make sure to check the version number.
12. mkdir ~/flickrfs/mount - use a mount point under your home directory or use sudo
13. mkdir ~/flickrfs/mount/stream
14. python flickrfs.py ~/flickrfs/mount
15. tail -f ~/.flickrfs/log

Installation and Configuration of LocalSolr.

A client of mine asked me to investigate localsolr which= extends Apache Solr server with the ability to perform text searches filtered on geographical distance from a given point specified by latitude and longitude. Here are the steps that I followed:

Download the Software

1. Downland Ant from http://apache.mirror99.com/ant/binaries/apache-ant-1.7.0-bin.tar.gz
2. Download Tomcat from http://mirror.nyi.net/apache/tomcat/tomcat-6/v6.0.16/bin/apache-tomcat-6.0.16.tar.gz
3. Download LocalLucene from http://downloads.sourceforge.net/locallucene/locallucene-r1.5.tar.gz
4. Download LocalSolr from http://downloads.sourceforge.net/locallucene/localsolr-r1.5.tar.gz
5. Download Solr-1.3-dev, via svn, to get the source code

   svn checkout http://svn.apache.org/repos/asf/lucene/solr/trunk apache-solr-1.3.dev

6. Uncompress all of the software packages.

Create Environment Variables

1. In Unix variants, update your .bash_login script to include the following. Of course, you'll need to set the variables correctly according to where you uncompressed the software. And don't forget to source the .bash_login file after you have changed it.

   export SUPPORT_DIR=/home/medined/support
   export LOCAL_LUCENE_HOME=$SUPPORT_DIR/locallucene-r1.5
   export LOCAL_SOLR_HOME=$SUPPORT_DIR/localsolr-r1.5
   export SOLR_HOME=$SUPPORT_DIR/apache-solr-1.3-dev
   export SOLR_CONFIG=/home/medined/.solr
   export TOMCAT_HOME=$SUPPORT_DIR/apache-tomcat-6.0.16
   
   export JAVA_OPTS="$JAVA_OPTS -Dsolr.solr.home=$SOLR_CONFIG"

Create Apache Solr War

1. cd $SOLR_HOME
2. ant dist

Configure Local Solr

1. cp -R $SOLR_HOME/example/solr $SOLR_CONFIG
2. cp $SOLR_HOME/libs/solr/apache-solr-1.3-dev.war $TOMCAT_HOME/webapps/solr.war mkdir $SOLR_CONFIG/lib
3. cp $LOCAL_LUCENE_HOME/dist/locallucene.jar $SOLR_CONFIG/lib
4. cp $LOCAL_SOLR_HOME/dist/localsolr.jar $SOLR_CONFIG/lib
5. cp $LOCAL_LUCENE_HOME/lib/gt2-referencing-2.3.1.jar $SOLR_CONFIG/lib
6. cp $LOCAL_LUCENE_HOME/lib/geoapi-nogenerics-2.1-M2.jar $SOLR_CONFIG/lib
7. cp $LOCAL_LUCENE_HOME/lib/jsr108-0.01.jar $SOLR_CONFIG/lib
8. Add the following to the end of $SOLR_CONFIG/conf/solrconfig.xml (just before the closing CONFIG tag:

     <updateRequestProcessor>
       <factory name="standard" class="solr.ChainedUpdateProcessorFactory" default="true">
         <chain class="com.pjaol.search.solr.update.LocalUpdateProcessorFactory">
           <str name="latField">lat</str>
           <str name="lngField">lng</str>
           <int name="startTier">9</int>
           <int name="endTier">17</int>
         </chain>
         <chain class="solr.LogUpdateProcessorFactory" >
          <!-- <int name="maxNumToLog">100</int> -->
         </chain>
         <chain class="solr.RunUpdateProcessorFactory" />
       </factory>
     </updateRequestProcessor>
   
    <requestHandler name="geo" class="com.pjaol.search.solr.LocalSolrRequestHandler">
       <!-- Custom latitude longitude fields, below are the defaults if not otherwise
       specified -->
        <str name="latField">lat</str>
        <str name="lngField">lng</str>
     </requestHandler>

9. Add the following to the fields tag of $SOLR_CONFIG/conf/schema.xml

   <field name="lat" type="sdouble" indexed="true" stored="true"/>
      <field name="lng" type="sdouble" indexed="true" stored="true"/>
      <dynamicField name="_local*" type="sdouble" indexed="true" stored="true"/>

Controlling Tomcat

• $TOMCAT_HOME/bin/startup.sh - this command starts Tomcat.
• tail -f $TOMCAT_HOME/logs/catalina.out - this command lets you watch Tomcat's output log.
• $TOMCAT_HOME/bin/shutdown.sh - this command stops Tomcat.

If you're lucky enough to be using Unix then combine the first two commands onto one line:

$TOMCAT_HOME/bin/startup.sh; tail -f $TOMCAT_HOME/logs/catalina.out

Importing Data Into Apache Solr

When LocalSolr is deployed into Tomcat, the default port is 8080. However, the Apache Solr import tools use a different hardcoded port. This causes me a but of angst until I realized that I could easily copy the SimplePostTool and change the port. So copy $SOLR_HOME/src/java/org/apache/solr/util/SimplePostTool.java, change the port specified on line 46 (see below) and compile it.

public static final String DEFAULT_POST_URL = "http://localhost:8080/solr/update";

I create a data file that looked like this

<add>
 <doc>
   <field name="id">01</field>
   <field name="name">HOUSE01</field>
   <field name="lat">39.36</field>
   <field name="lng">-77.4027</field>
   <field name="text">zxy</field>
 </doc>
 <doc>
   <field name="id">02</field>
   <field name="name">HOUSE02</field>
   <field name="lat">38.36</field>
   <field name="lng">-77.4027</field>
   <field name="text">zxy</field>
 </doc>
</add>

Then I simply executed the SimplePostTool program passing it the name of the data file as the program argument.

Apache Solr Admin Screen

With Tomcat running, you should be able to connect with http://localhost:8080/solr/admin/

Local Solr GIS Query

With Tomcat running, you should be able to execute a GIS-based query by connecting to http://localhost:8080/solr/select?&qt=geo&lat=38.8700&long=-77.4027&q=zxy&radius=1. The q=zxy tells Apache Solr to return all documents. The lat and long parameters indicate the center of the circle to search using decimal degrees. While the radius parameter indicates the radius, in miles, of the circle.

Good Luck!

Struts Error/Exception: No getter method available for property name for bean under name

Recently I was using Struts v1.x and I needed to display a select list. The information about how to use the html:optionsCollection tag was sketchy. I was able to get the tag working this way:

On the JSP page, I added this inside the html:form tag:

<html:select property="selectedEquipment">
  <html:optionsCollection property="equipment" label="name" value="id"/>
</html:select>

Then I create a Java class called SelectOption like this:

package com.codebits.struts;

public class SelectOption {

    String id;
    String name;

    SelectOption() {
    }

    SelectOption(final String _id, final String _name) {
        setId(_id);
        setName(_name);
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

Next I created an ActionForm bean:

package com.codebits.struts;

public class AddForm extends ActionForm {

    private Integer           selectedEquipment = null;

    private List              equipment         = new ArrayList();
    
    public Integer getSelectedEquipment() {
        return selectedEquipment;
    }
    
    public void setSelectedEquipment(Integer selectedEquipment) {
        this.selectedEquipment = selectedEquipment;
    }
    
    public List getEquipment() {
        return equipment;
    }
    
    public void addEquipment(final String equipmentId, final String emtEquipmentId) {
        this.equipment.add(new SelectOption(equipmentId, emtEquipmentId));
    }
    
    public void setEquipment(List equipment) {
        this.equipment = equipment;
    }

}

And finally, in my action class, I prepopulated the ActionForm bean:

  AddForm addForm = (AddForm)form;
  addForm.addEquipment("1", "AAA");
  addForm.addEquipment("2", "BBBB");

How to Format Dates in Ruby & Rails

I have seen several techniques used to format dates in blogs and forums threads. This is the technique that worked for me.

1. Create a file called config/initializers/date_formats.rb with the following contents:

   ActiveSupport::CoreExtensions::Time::Conversions::DATE_FORMATS.merge!(
    :date => '%m/%d/%Y',
    :date_time12  => "%m/%d/%Y %I:%M%p",
    :date_time24  => "%m/%d/%Y %H:%M"
   )

2. Set the date variable:

   @cacheExpiresAt = 10.minutes.ago

3. Format the date:

   <%= @cacheExpiresAt.to_s(:date_time12) %>

The :date_time12 parameter to the to_s method simply chooses the formatting from the DATE_FORMATS hash. This technique lets you centralize all date formatting in your application.

Workaround For Minor Hibernate Bug - Blank hibernate.default_schema Property Adds Period

We are using the hibernate.default_schema property with Oracle in production. Today I tried to use HSQL for our integration tests. When I blanked the hibernate.default_schema property, Hibernate tried to create tables named like '.FOO' - notice that pesky leading period! You can workaround this issue for HSQL by setting hibernate.default_schema to 'sa'.

ActiveRecord Without Rails, Refined

About a year ago Aizat Faiz, a software developer in Malaysia wrote a blog entry about using ActiveRecord outside of the Rails framework. This is definitely something that I want to do for a variety of reasons, the simplest being that I'd like to write utility programs that access the database without needing to also write a user interface for them.

In any case, Aizat wrote a nice understandable entry. However, he seemed to make the assumption that developers using ActiveRecord without Rails would not have a Rails application at all. This is not the case for me.

If your utility script resides in the top level directory, then you can use your already existing database.yml file instead of creating a new one. Just use the following as a guide.

dbconfig = YAML::load(File.open('config/database.yml'))
ActiveRecord::Base.establish_connection(dbconfig['development'])

Also, there is no reason to have your models in your script when they are already in your Rails application. You can load existing modesl like this:

require 'app/models/event.rb'
require 'app/models/senator.rb'

That's it. Very simple and no duplication of code.

How To Add a Datasource To Tomcat Server.XML File.

... ... factory oracle.jdbc.pool.OracleDataSourceFactory validationQuery select 1 from dual url jdbc:oracle:thin:USERNAME/PASSWORD@localhost:1521:XE maxActive 4 maxWait 5000 driverClassName oracle.jdbc.OracleDriver maxIdle 2 ... ..

How to Display the Current Loadpath In Ruby

This is a simple bit, but useful.

C:\data\ruby\PoliticalBits]ruby -e 'puts $:'
C:/data/support/ruby-186-26/lib/ruby/site_ruby/1.8
C:/data/support/ruby-186-26/lib/ruby/site_ruby/1.8/i386-msvcrt
C:/data/support/ruby-186-26/lib/ruby/site_ruby
C:/data/support/ruby-186-26/lib/ruby/1.8
C:/data/support/ruby-186-26/lib/ruby/1.8/i386-mswin32

Ubuntu Sound not working on HP Pavillion dv9000 Using the ICH8 Family

I spent quite a bit of time researching my problem and found the solution at: http://www.mail-archive.com/ubuntu-bugs@lists.ubuntu.com/msg549082.html Let me copy the relevant bits:

cd /tmp
wget ftp://ftp.alsa-project.org/pub/driver/alsa-driver-1.0.16rc2.tar.bz2
tar -jxvf alsa-driver-1.0.16rc2.tar.bz2
cd alsa-driver-1.0.16rc2/
./configure --with-cards=hda-intel && make
sudo make install
sudo cp ./modules/snd-hda-intel.ko /lib/modules/$( uname -r )/ubuntu/media/snd-hda-intel/
sudo depmod -a
sudo reboot

Make sure that you check the ftp://ftp.alsa-project.org/pub/driver directory to see if a new version has been released.

Failed dependencies Message (glibc, libaio, /bin/sh) Installing Oracle XE on Ubuntu

I downloaded the RPM for Oracle XE for Linux and tried to install it without success. Here is my result:

medined@bashi:~/Desktop$ sudo rpm -ivh oracle-xe-10.2.0.1-1.0.i386.rpm
error: Failed dependencies:
        glibc >= 2.3.2 is needed by oracle-xe-10.2.0.1-1.0.i386
        libaio >= 0.3.96 is needed by oracle-xe-10.2.0.1-1.0.i386
        /bin/sh is needed by oracle-xe-10.2.0.1-1.0.i386

I did some research but could not find a way to update glibc. However, I did run into an apt-get repository mentioned at http://www.oracle.com/technology/tech/linux/install/xe-on-kubuntu.html. I followed the instructions on that page and had Oracle working within minutes.

How To Turn Off the Keyboard Beep in Fedora & Gnome

The beep that occurs when working at the command-line or inside a text editor is not controlled by Gnome settings. Instead, it is controlled by a terminal profile. Open a terminal, right-click and select 'Edit Current Profile'. Then uncheck the 'Terminal Bell' checkbox.

Set Mode Failed Invalid Argument for Wireless Linux Configuration

I am trying to get the wireless networking working on Fedora installed into my HP dv9000 laptop when I ran into this error. The problem was caused because the system was trying to set the wireless mode to be AUTO instead of MANAGED. Update: I ran the iwlist command and found out the information that I needed to get the wireless connection working.

[root@localhost network-scripts]# /sbin/iwlist wlan0 scan
wlan0     Scan completed :
          Cell 01 - Address: 00:16:B6:1F:2B:CC
                    ESSID:"ECLECTIC_CONSULTING"
                    Mode: Master
                    Channel: 6
                    Frequency:2.437 GHz (Channel 6)
                    Quality=100/100  Signal level=-33 dBm  Noise level=-96 dBm
                    Encryption key:on
                    Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 18 Mb/s
                              24 Mb/s; 36 Mb/s; 54 Mb/s; 6 Mb/s; 9 Mb/s
                              12 Mb/s; 48 Mb/s
                    Extra:tsf=000000007b651a75

The important bits are bolded - the ESSID, Mode, and Channel. Use the system-config-network command to set the properties of the wlan0 connection. I'm publishing this blog from my laptop via a wireless connection so I know my system is now working correctly.

Super Simple Embedded Groovy

Here is the simplest possible Java program that runs a Groovy Script (as far as I know).

package affy.groovy.driver;

import groovy.lang.GroovyShell;

public class GroovyDriver {
 public static void main(String[] args) {
  new GroovyShell().evaluate("println('Hello World');");
 }
}

You'll need three jar files to make this program compile and execute:

  groovy-1.0.jar
  asm-2.2.jar
  antlr-2.7.5.jar

Simple Design And Testing Conference - York, PA - Nov 30 to Dec 2

Time is speeding by but there is still time to register for this intellectually stimulating conference. Find out more information on their website at http://www.sdtconf.com/. Hope to see you there!

WARNING: Cound not start ClientFactory - FIXED!

This warning also says that the javax.naming.NoInitialContextException was caused by a missing java.naming.factory.initial parameter. I am seeing this error while running an embedded ServiceMix engine. It is easy to fix however. You need to add two files to the classpath. jndi.properties

java.naming.factory.initial = org.apache.xbean.spring.jndi.SpringInitialContextFactory

jndi.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<beans>
  <bean id="jndi" class="org.apache.xbean.spring.jndi.DefaultContext">
    <property name="entries">
      <map/>
    </property>
  </bean>
</beans>

Dynamic Datasource via Spring using HotSwappableTargetSource

My goal was to create a jar file that encapsulates an application domain objects and their persistence. However, I wanted to provide a way for the calling routines to change the datasource as needed. This feature was not intended to allow swapping datasources in the middle of an application's running but rather to allow the jar file to be ignorant of the test and production configurations. We'll start by creating hypersonic.properties:

jdbc.driver=org.hsqldb.jdbcDriver
jdbc.url=jdbc:hsqldb:.
jdbc.username=sa
jdbc.password=

hibernate.dialect=org.hibernate.dialect.HSQLDialect
hibernate.hbm2ddl.auto=update

Then create hypersonicContext.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
 
 <bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
  <property name="location" value="hypersonic.properties"/>
 </bean>
 
 <import resource="beanDefinition.xml"/> 
 
</beans>

I had developed this technique of having a separate set of property file and Spring configuration file for each database that I wanted to access. I've included it in this entry as a side note. The beanDefinition.xml file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
 
 <!--
 This is the bean that the application 'sees' as the datasource. Underneath the covers, the real
 datasource is swapped as needed.
 -->
 <bean id="dataSource" class="org.springframework.aop.framework.ProxyFactoryBean">
  <property name="targetSource" ref="swappableDataSource"/>
 </bean>
 
 <!--
 This is a magic bean from Spring that allows the underlying (or real) datasource to be
 swapped.
 -->
 <bean name="swappableDataSource" class="org.springframework.aop.target.HotSwappableTargetSource">
  <constructor-arg ref="dummyDataSource"/>
 </bean>
 
 <!-- 
 This dummy datasource is here just to show that you can start off with zero information about the 
 datasource. Later, as the program is running and the datasource information becomes known you 
 can hot swap to the right datasource. 
 -->
 <bean id="dummyDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"/>
 
 <!--
 This datasource shows how to use properties set by an instance of PropertyPlaceholderConfigurer.
 You can define as many of these types of datasources as you'd like. Switch between them 
 -->
 <bean id="defaultDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
  <property name="driverClassName" value="${jdbc.driver}" />
  <property name="url" value="${jdbc.url}" />
  <property name="username" value="${jdbc.username}" />
  <property name="password" value="${jdbc.password}" />
 </bean>
 
</beans>

The comments in the XML should explain what is happening. Now comes the DataSourceFactory class which contains the magic.

package factory;

import java.util.HashMap;
import java.util.Map;

import javax.sql.DataSource;

import org.apache.commons.dbcp.BasicDataSource;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.aop.target.HotSwappableTargetSource;
import org.springframework.context.ApplicationContext;
import org.springframework.util.Assert;

import com.judoscript.JudoEngine;

/**
 * This class allows programs to dynamically change the dataSource 
 * they use. For example, Spring can be configured to know about
 * three beans - defaultDatasource, testDataSource, and 
 * prodDataSource. During development,the defaultDataSource is used 
 * because the getDataSource() method is called without any other 
 * specifications. When the code is deployed to the test environment, that
 * process creates a system property called 'datasource.spring.beanname' 
 * set to testDataSource. Spring's HotSwappableTargetSource
 * feature is used to dynamically switch to the testDataSource bean 
 * previously defined in the Spring configuration files.
 * 
 * 1. Read system property (datasource.script) to get judoscript to execute.
 * 
 * 2. Read system property (datasource.script.filename) to get name 
 * of judoscript file to execute.
 * 
 * 3. Read system property (datasource.spring.beanname) to get name of 
 * Spring bean to load.
 */
public class DataSourceFactory {

 private Log log = LogFactory.getLog(getClass());

 private ApplicationContext ctx = null;

 public DataSourceFactory(ApplicationContext _ctx) {
  super();
  this.ctx = _ctx;
 }

 public DataSource getDataSource() {
  DataSource realDataSource = null;

  String dataSourceScript = System.getProperty("datasource.script");
  String judoScriptFileName = System.getProperty("datasource.script.filename");
  String springBeanName = System.getProperty("datasource.spring.beanname");
  if (dataSourceScript != null) {
   realDataSource = helperFromJudoScriptString(dataSourceScript);
   log.debug("defining DataSource from JudoScript string, via system property.");
  } else if (judoScriptFileName != null) {
   realDataSource = helperFromJudoScriptFile(judoScriptFileName);
   log.debug("defining DataSource from JudoScript script, via system property, named [" + judoScriptFileName + "].");
  } else if (springBeanName != null) {
   realDataSource = (DataSource) ctx.getBean(springBeanName);
   log.debug("defining DataSource from Spring bean, via system property, named [" + springBeanName + "].");
  } else {
   realDataSource = (DataSource) ctx.getBean("defaultDataSource");
   log.debug("defining DataSource from default Spring bean [defaultDataSource] in Spring configuration.");
  }
  return swapToDataSource(realDataSource);
 }

 public DataSource getDataSourceFromSpringBean(final String name) {
  DataSource realDataSource = (DataSource) ctx.getBean(name);
  log.debug("defining DataSource from Spring bean named [" + name + "].");
  return swapToDataSource(realDataSource);
 }
 
 public DataSource getDataSourceFromJudoScriptString(final String script) {
  DataSource realDataSource = helperFromJudoScriptString(script);
  log.debug("defining DataSource from JudoScript string.");
  return swapToDataSource(realDataSource);
 }
 
 public DataSource getDataSourceFromJudoScriptFile(final String filename) {
  DataSource realDataSource = helperFromJudoScriptFile(filename);
  log.debug("defining DataSource from JudoScript file called [" + filename + "].");
  return swapToDataSource(realDataSource);
 }
 
 public DataSource getDataSourceFromDbcpBasicDataSource(final String driverClassName, final String url, final String username, final String password) {
  BasicDataSource realDataSource = new BasicDataSource();
  realDataSource.setDriverClassName(driverClassName);
  realDataSource.setUrl(url);
  realDataSource.setUsername(username);
  realDataSource.setPassword(password);
  return swapToDataSource(realDataSource);
 }
 
 private DataSource swapToDataSource(final DataSource realDataSource) {
  Assert.notNull(realDataSource, "Error defining the real dataSource.");
  HotSwappableTargetSource swapper = (HotSwappableTargetSource) ctx.getBean("swappableDataSource");
  swapper.swap(realDataSource);
  return (DataSource) ctx.getBean("dataSource");
 }

 private DataSource helperFromJudoScriptFile(final String filename) {
  Map sysprops = new HashMap();
  String[] jeArgs = {};
  JudoEngine je = null;
  DataSource rv = null;

  // define the datasource via judoscript.
  try {
   je = new JudoEngine();
   je.putBean("root", new HashMap());
   je.runScript(filename, jeArgs, sysprops);
   Map root = (Map) je.getBean("root");
   rv = (DataSource) root.get("dataSource");
  } catch (Throwable e) {
   throw new RuntimeException(e);
  }

  return rv;
 }

 private DataSource helperFromJudoScriptString(final String script) {
  Map sysprops = new HashMap();
  String[] jeArgs = {};
  JudoEngine je = null;
  DataSource rv = null;

  // define the datasource via judoscript.
  try {
   je = new JudoEngine();
   je.putBean("root", new HashMap());
   je.runCode(script, jeArgs, sysprops);
   Map root = (Map) je.getBean("root");
   rv = (DataSource) root.get("dataSource");
  } catch (Throwable e) {
   throw new RuntimeException(e);
  }

  return rv;
 }

}

Finally, all of the pieces come together in an example program.

package drivers;

import java.sql.Connection;

import javax.sql.DataSource;

import factory.DataSourceFactory;

import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class DatabasePopulationServiceDriver {

 public static void main(String[] args) {
  
  ApplicationContext ctx = new ClassPathXmlApplicationContext(new String[] {"hypersonicContext.xml"});

  try {
   DataSourceFactory dataSourceFactory = new DataSourceFactory(ctx);

                        // If the getDataSource() method is used then
                        // start the program with -D to define the system
                        // property which controls the data source.
   DataSource dataSource = dataSourceFactory.getDataSource();

                        // Or one of the more specific method can be used.
                        DataSource dataSource = dataSourceFactory.getDataSourceFromSpringBean("prodDataSource");

   Connection connection = dataSource.getConnection();
   connection.close();

  } catch (Exception e) {
   e.printStackTrace();
  } finally {
   System.out.println("Done.");
  }
 }

}

Automatic Directory Creation via Spring Application Context File

Spring lets you define property editors that convert from a text representation to an object representation. We can hook into this feature to automate directory creation like this:

public class DirectoryEditor extends PropertyEditorSupport {

 public void setAsText(String textValue) {
  File f = new File(textValue);
  if (!f.exists()) {
   if (!f.mkdirs()) {
    throw new ConfigurationException("Unable To Create Directory: [" + textValue + "]");
   }
  } else if (!f.isDirectory()) {
   throw new ConfigurationException("File Is Not a Directory: [" + textValue + "]");
  }
  setValue(new Directory(textValue));
 }
 
}

Then create a bean definition so that Spring knows about it:

  <bean id="customEditorConfigurer"
class="org.springframework.beans.factory.config.CustomEditorConfigurer">
    <property name="customEditors">
      <map>
        <entry key="com.twintechs.spring.Directory">
          <bean id="directoryEditor" class="com.twintechs.spring.DirectoryEditor" />
        </entry>
      </map>
    </property>
</bean>

Then create a Directory class:

public class Directory {

 private String name;

 public Directory(String _name) {
  super();
  this.name = _name;
 }

 public String getName() {
  return name;
 }

 public void setName(String name) {
  this.name = name;
 }
 
}

The last step is to use the Directory class in place of the String class in any Java objects.

Extracting Dependancy Lists from Maven POM Files

Recently a client of mine wanted to know if they were including the same dependency versions in their subprojects. They had 12 modules feeding into one maven-based build so I was inefficient to check the dependencies by hand. After running this program, copy the output into a program to sort it.

package com.affy.pom;

import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Set;
import java.util.TreeSet;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.xpath.XPath;
import javax.xml.xpath.XPathConstants;
import javax.xml.xpath.XPathExpression;
import javax.xml.xpath.XPathExpressionException;
import javax.xml.xpath.XPathFactory;

import org.w3c.dom.Document;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.SAXException;

public class PomAnalysisDriver {

 private static XPathFactory xpathFactory = XPathFactory.newInstance();

 XPathExpression artifactIdExpr = null;

 XPathExpression dependencyExpr = null;

 XPathExpression depGroupIdExpr = null;

 XPathExpression depArtifactIdExpr = null;

 XPathExpression depVersionExpr = null;

 public static void main(String[] args) throws Exception {
  try {
   List pomFiles = new ArrayList();

   PomAnalysisDriver pad = new PomAnalysisDriver();
   pad.init();
   pad.searchForPomFiles("[ROOT_DIR]", pomFiles);
   pad.execute(pomFiles);
  } finally {
   System.out.println("Done.");
  }
 }

 public void init() throws XPathExpressionException {
  XPath xpath = xpathFactory.newXPath();
  artifactIdExpr = xpath.compile("/project/artifactId/text()");
  dependencyExpr = xpath.compile("/project/dependencies/dependency");
  depGroupIdExpr = xpath.compile("./groupId/text()");
  depArtifactIdExpr = xpath.compile("./artifactId/text()");
  depVersionExpr = xpath.compile("./version/text()");
 }

 public void searchForPomFiles(final String dir, final List pomFiles) {
  File initialDir = new File(dir);
  if (initialDir.isDirectory() == false) {
   return;
  } else {
   String[] files = initialDir.list();
   for (String filename : files) {
    String suffix = "";
    if (dir.endsWith("\\") == false && dir.endsWith("/") == false) {
     suffix = "\\";
    }
    File childDir = new File(dir + suffix + filename);
    if (childDir.isDirectory() == true) {
     searchForPomFiles(dir + suffix + filename, pomFiles);
    } else {
     if (filename.equals("pom.xml")) {
      pomFiles.add(childDir.getAbsolutePath());
     }
    }
   }
  }
 }

 public void execute(final List pomFiles) throws SAXException, IOException,
 ParserConfigurationException, XPathExpressionException {
  for (String pomFilename : pomFiles) {
   File f = new File(pomFilename);
   Document document;

   DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
   DocumentBuilder builder = factory.newDocumentBuilder();
   document = builder.parse(f);

   // Node projectNode = document.getDocumentElement();
   // NodeList children = projectNode.getChildNodes();

   String projectName = (String)artifactIdExpr.evaluate(document, XPathConstants.STRING);
   
   List dependencies = new ArrayList();

   NodeList nodes = (NodeList)dependencyExpr.evaluate(document, XPathConstants.NODESET);
   for (int i = 0; i < nodes.getLength(); i++) {
    Node dependency = nodes.item(i);
    String depGroupId = (String) depGroupIdExpr.evaluate(dependency,
                                          XPathConstants.STRING);
    String depArtifactId = (String) depArtifactIdExpr.evaluate(dependency,
                                          XPathConstants.STRING);
    String depVersion = (String) depVersionExpr.evaluate(dependency,
                                          XPathConstants.STRING);
    String depId = "/" + depGroupId + "/" + depArtifactId + "/" + 
                                          depVersion + "/" + projectName;
    dependencies.add(depId);
   }

   for (String s : dependencies) {
    System.out.println(s);
   }
  }
 }

}

Setting Up a Ubuntu Perforce Server

This is a step-by-step guide to installing and configuring Perforce on a Ubuntu server. It assumes that you have already created a Basic Ubuntu Server. There are several sections below:

• Download Perforce
• Configure Perforce
• Run Perforce Server For the First Time
• Setup Perforce As Bootup Service
• ToDo
• Resources

Download Perforce

1. Use Lynx to download the P4D software.

     lynx http://www.perforce.com/perforce/downloads/linux26x86.html
   

2. Enter Y to allow the cookie from Perforce.com.
3. Press the Down Arrow until the Downloads link is selected. Then press Enter.
4. Press the Down Arrow until the Linux Kernel 2.6 for x86 link is selected. Then press Enter.
5. Press the Down Arrow until the Download link is selected. Then press Enter.
6. Press D to start the download.
7. Press the Down Arrow to highlight the ''Save to disk'' link. Then press Enter.
8. Use the default filename of `p4d`.
9. Press q, then press Enter to exit Lynx.
10. Use Lynx to download the P4 software by retrace the steps above expect selecting to download `P4` instead of `P4D`.

Configure Perforce

This information is a reprise of the [http://www.perforce.com/perforce/doc.072/manuals/p4sag/index.html Perforce Administration Guide] along with some additional information.

1. Download the `daemon` utility. This utility allows `p4d` to be run by the `perforce` user instead of `root`.

     sudo apt-get daemon
   

2. Connect to the directory where the `p4` and `p4d` files where downloaded (probably the home directory).

     cd ~
   

3. Make the `p4` and `p4d` files executable.

     chmod +x p4 p4d
   

4. Copy the Perforce executables to `/usr/local/bin` which is already in the system PATH environment variable.

     sudo mv p4 /usr/local/bin
     sudo mv p4d /usr/local/bin
   

5. Create a group for Perforce files.

     sudo addgroup p4admin
   

6. Create a user for Perforce administrative work.

     sudo adduser perforce
   

7. Use `visudo` to give the perforce user account the ability to use `sudo`. Add the following line at the end o f the file.

     perforce ALL = ALL
   

8. Log off your default user account.
9. Log in using the `perforce` account.
10. Create a directory to hold the repository.

      sudo mkdir /perforce_depot
      sudo chown perforce:p4admin /perforce_depot  
    

11. Create a directory to hold Perforce log files.

      sudo mkdir /var/log/perforce
      sudo chown perforce:p4admin /var/log/perforce
    

12. Add the following lines to the end of /etc/profile. These settings will be used by client programs run on the Linux server - not by the Perforce server.

      # Perforce Settings
      export P4JOURNAL=/var/log/perforce/journal
      export P4LOG=/var/log/perforce/p4err
      export P4PORT=localhost:1666
      export P4ROOT=/perforce_depot
      export P4USER=perforce
    

13. Load the Perforce settings.

      source /etc/profile
    

14. Create a Perforce group to limit resource usage using `sudo p4 group developers` to set the following values. See http://kb.perforce.com/P4dServerReference/Performance/ViewsProtect..Maxlocktime for more information.

      Group: developers
      MaxResults:  50000
      MaxScanRows: 250000
      MaxLocktime: 30000
      Timeout:     4320
      Subgroups:
      Users:
        developer
    

Setup Perforce As Bootup Service

1. Connect to the initialization control directory.

     cd /etc/init.d
   

2. Create the Perforce control script using `sudo vi perforce`.

     #!/bin/sh -e
   
     export P4JOURNAL=/var/log/perforce/journal
     export P4LOG=/var/log/perforce/p4err
     export P4ROOT=/perforce_depot
     export P4PORT=1666
   
     PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"
   
     . /lib/lsb/init-functions
   
     p4start="p4d -d"
     p4stop="p4 admin stop"
     p4user=perforce
   
     case "$1" in
     start)
       log_action_begin_msg "Starting Perforce Server"
       daemon -u $p4user $p4start;
       ;;
   
     stop)
       log_action_begin_msg "Stopping Perforce Server"
       daemon -u $p4user $p4stop;
       ;;
   
     restart)
       stop
       start
       ;;
   
     *)
       echo "Usage: /etc/init.d/perforce (start|stop|restart)
       exit 1
       ;;
   
   esac
   
   exit 0
   

3. Integrate the Perforce control script into the boot process.

     sudo update-rc.d perforce defaults
   

4. Goto your home directory then use the control script to start the Perforce server.

     cd ~
     sudo /etc/init.d/perforce start
   

5. Use the control script to restart the Perforce server.

     sudo /etc/init.d/perforce restart
   

Run Perforce Server For the First Time

Before Perforce can be used by a team there are two housekeeping task that need to be done - creating the journal and closing a security hole.

1. Set the Perforce environment variables.

     source /etc/profile
   

2. Start the perforce server. The command line is short because all of the settings are provided by the variables exported by /etc/profile.

     sudo p4d –d
   

3. Create the journal.

     sudo p4d –jc
   

4. By default, all users connected to Perforce are administrators. However, that level of security is not acceptable for a normal development environment. Run the `protect` option to switch out of the default security mode. For now, accept the default protections by typing `:wq` and pressing Enter.

     sudo p4 protect
   

5. Stop the Perforce server.

     sudo p4 admin stop
   

TODO

* Journalling
* checkpointing
* backups

Resources

* Perforce's public depot
* http://kb.perforce.com/AdminTasks/InstallAndUpgrade/RunningAPerf..InetdOnUnix How to start `p4d` via `inetd` instead of a startup script.