Saturday, 4 December 2010

Hammock Driven Development

I would love to be able to claim credit for that phrase. But I can't. It's actually the title of a talk that Rich Hickey gave at the first Clojure Conj in which he 'rants' about the importance of really thinking about problems. These are my notes on the talk.

It's all very well having methodologies and processes for developing great software. But it's even more important to really understand the problem you are trying to solve. Bugs discovered in production are more expensive to fix than bugs discovered in the development phase. The less code that exists, the easier it is to fix problems. By extension, it's easiest to fix problems when no code exists at all. Time spent critically thinking about potential solutions enables you to identify errors that, if unresolved, would ultimately lead to bugs in the code.

"Most of the biggest problems in software are problems of misconception."

So what do you do?
  1. State the problem. Say it out loud or write it down. You need to be clear about what problem you are trying to solve. And if you are not trying to solve a problem then what exactly are you doing?
  2. Understand the problem. Read around it. Are there related problems? What don't you know? What questions do you have?
Not everything is awesome. Look for problems in your own solutions. Look for problems in the solutions proposed by others. By constantly challenging the solution you can refine it and improve it.

So where does the hammock come into this? Well, Mr. Hickey contrasts what he calls the waking mind and the background mind. The waking mind is good at analysing, finding problems, and identifying 'local maxima' - polishing an idea until it shines. The background mind makes the giant leaps of intuition to generate completely different ideas. The key is to utilize both aspects of the mind.

When you have a problem to solve, use the waking mind to provide the background mind with data. Read around the problem, think about it, draw pictures, write a song, whatever. Use your waking mind to cram as much information as you can find about the problem into your head.

The next step is to hit the hammock. Take some time to lie down, close your eyes, and think. Rich refers to this as "mind's eye time". The important thing is to step away from the computer, and away from all distractions. You need time to process all the input, to think about it and analyse it from as many angles as you can come up with.

The time spent on the hammock is time spent reinforcing everything that you have input. Then you hand over to the background mind to let it come up with the solution. It probably won't be instantaneous. You should at least expect to wait overnight, and it may even take months. But eventually, Deep Thought will speak. Probably while you're in the shower.

I always enjoy watching videos of Rich Hickey speaking. He comes across as someone who knows his subject inside out. Having watched this particular video I now understand why. He clearly thinks very deeply about the problems he is trying to solve. This is something that I would like to emulate.

Now I'm going to go find a hammock to think this over some more.

Sunday, 17 October 2010

Kindle Rocks!

Yes, I suck.

My Kindle arrived yesterday morning, and it is already my new favourite gadget. Not that I have any other newer gadgets, but you know what I mean. It rocks.

As soon as I powered it up and started reading through the supplied manual, I felt comfortable. One of the design goals was to make the Kindle disappear, so that you forget you are using it. Well, they succeeded! The e-Ink screen looks just like a page in a book and leads to a really immersive experience.

From my point of view Kindle has 2 killer features:
  1. The ability to highlight passages of text in a book for future reference. I love this. The ability to add notes is also pretty cool.
  2. Instant gratification. I love being able to buy books and have them instantly available to read. Sony missed a trick with their e-reader. I probably would have bought one if it had similar wireless capability.
In the space of 1 day I have become a complete Kindle convert. 

But what about my qualms about DRM, and Amazon lock in? Well, all I can say is that pragmatism trumped idealism. I was in the market for an e-reader, and Amazon provide the best all round e-reading experience on the market.

Besides, I'm pretty sure the DRM can be hacked if necessary. Hopefully it won't be necessary.

Come on Amazon, come on book publishers. Free the books. Make it even easier for me to give you my money.

Wednesday, 13 October 2010

Converting Meditation Helper to Scala: Aborted

I have aborted this project due to the risk involved in re-writing the application from scratch. It has taken a lot of hard work to eradicate the little bugs in the code - there's nothing like having a couple of thousand users to flush out obscure bugs! It's only really with version 1.3.0 that I feel that I have a really solid application. I don't want to throw away all that hard work, and possibly introduce regressions in future releases.

So, I've decided to keep writing Meditation Helper in Java. For my next app (whatever that may be) I will try Scala.

Sunday, 19 September 2010

Converting Meditation Helper to Scala: I Hate Ant. And Eclipse.

I hate Ant. I really do. So much that I decided to revisit my strategy for this project. At first I tried to configure Maven, but that proved really difficult. I like Maven a lot, but the combination of the Android plugin, the Scala plugin, and the Proguard plugin seems really difficult to configure at the minute.

And so I have revisited sbt. I now have a working build using sbt. The next problem is to integrate that nicely with an IDE. I have tried to make this work well with Eclipse but, well I may as well say it - I hate Eclipse. I really do.

So where am I - sbt works well, and is probably the simplest solution for building an Android project in Scala. The Eclipse Scala plugin is a bit flaky. So - I've read good things about Intellij. The community edition will do Scala via a plugin. It won't do Android, but that's fine - I can handle all the Android stuff either from the command line or using sbt. All I need is a good Scala editor.

I will use http://github.com/mpeltonen/sbt-idea-plugin to create the project files for Intellij. Fingers crossed...

Well that went very smoothly. After a weekend of wrestling with Eclipse, could I have found a winner in Intellij??

Yes.

It has taken me all weekend. I have torn out most of my hair. But I now have a working development environment to re-write my app in Scala. I am using:
  • Scala 2.8.0
  • Sbt 0.74
  • JUnit 4 
  • Intellij 9.0.3
All builds etc are run through the sbt console. I only use Intellij for it's Scala editing capabilities, with Android added as a standard jar library. Any Android stuff I can do directly using the tools from the command line.

Hallelujah. My work here is done. Now I can really write some code.

Tomorrow.

Monday, 13 September 2010

Converting Meditation Helper to Scala: Creating the Scala Project

Now that I have managed to install both the Scala IDE and Android plugins into the same instance of Eclipse, the next stage is to create a new Android project for the Scala version. To do that, I'm going to use the Android plugin for sbt, described at http://zegoggl.es/2009/12/building-android-apps-in-scala-with-sbt.html.
  • First things first - I need a version of Scala installed outside Eclipse. I will use the IzPack installer from http://www.scala-lang.org/downloads.
  • That was easy. Once it was installed all I had to do was add the ~/scala/bin directory to my PATH.
  • Now to follow the instructions at zegoggles to install sbt.
  • All seems good so far. Now to generate the project using the android-plugin listed on that page, and the instructions.
  • ../android-plugin/script/create_project MeditationHelperScala com.nwalex.meditation.scala --api-level 7
  • I've used a different package name so that I can install this version side by side with the current version on my phone. I will need to change this before I deploy the new version to the market.
  • Ah, that worked well but doesn't have Eclipse integration. 
Let's try another approach.
  • Using the instructions at https://www.assembla.com/wiki/show/scala-ide/Developing_for_Android I...
  • Created a new Android project in Eclipse using the wizard.
  • Created the Ant files using: android update project --target 1 --path .
  • Followed the directions to customize the ant file and the rest of the instructions and...
  • IT WORKS!!!
To stop Eclipse and Ant getting in the way of each other, I changed the build.properties so that the Ant generated classes go into gen-ant, and builds into bin-ant.

With the external build tool configuration I can now edit the files in Eclipse using the Scala perspective, and run ant to do the build and install to the emulator.

Now I can get cracking with the conversion.

Sunday, 12 September 2010

Converting Meditation Helper to Scala: Configuring Dev Environment

Now that Meditation Helper is fairly mature, I have decided to do a complete re-write in Scala. There are 2 reasons for doing this:
  1. The current code is a bit messy. Since I was learning how to develop for Android while writing the application, I made some mistakes early on that are making it difficult to add new features in an elegant way. I want to fix those mistakes.
  2. I want to learn Scala.
If it were possible to write Android applications in Clojure, then I would use Clojure. However, it appears that the dynamic nature of Clojure makes it impossible to run on Dalvik, since Dalvik is not Java (despite what Oracle thinks). It seems that Scala does not have this problem. It is a statically typed language that compiles directly to byte code.

I am going to record my progress here. I have no idea how it will turn out.

Approach
I intend to create a brand new Android project, then copy across the various resource files (GUI layouts etc). The new version will therefore look exactly the same as the Java version, and will use the same database tables to read from and write to. So the front end and back end will remain the same, with everything in between re-written in Scala.

Aim
Once I have a fully working version of the app written in Scala I want to compare the performance and the size of the package.

Notes

  • Ok, that's not a great start. I just tried to install the Scala IDE from http://www.scala-ide.org/. The installation seemed to go ok, however, it seemed to wipe out the Android plugin! The app wouldn't compile because the builders had disappeared. So, having reverted to a previous version of my .eclipse directory, I will try again by creating a new workspace, and installing the Scala plugins. Then I will switch back to the old workspace and check whether or not the app will compile.
  • The Scala plugin really doesn't seem to want to install. Downloading Eclipse Classic 3.5.2 to try with a fresh Eclipse installation.
  • That seemed to work. To be fair it does say at http://www.scala-ide.org/ that Eclipse Classic is a requirement. I was running one of the other Eclipse builds - Eclipse for Java developers I think. Now to re-install the Android plugin.
  • Had to add http://download.eclipse.org/releases/galileo/ to the list of update sites in Eclipse in order to resolve the Android dependencies. And even then it still doesn't work. Now neither Scala or Android appear to be available. This is turning into a yak shaving exercise.
  • Woo hoo! I had the wrong version of Eclipse classic - 3.5.0 instead of 3.5.2. 
So in summary, to have Scala and Android pluging in the same IDE I needed to:
  • Install Eclipse Classic 3.5.2
  • Add http://download.eclipse.org/releases/galileo/ to the list of available download sites
  • Add http://download.scala-ide.org/nightly-update-master-2.8.0.final to the list of available download sites and install the Scala IDE
  • Add https://dl-ssl.google.com/android/eclipse/ and install the Android plugins.
I hope things go more smoothly from now on...

Saturday, 14 August 2010

My First Android Application

I've just pushed the first version of my first Android application to the market:

http://meditation-helper.blogspot.com/2010/08/version-050-ugly-baby-release.html

It's a bit rough round the edges, but works pretty well. I've been using it every day for the last couple of months. I wonder if anyone will use it...

Thursday, 12 August 2010

The Problem with Kindle

I refuse to voluntarily lock myself in to Amazon’s proprietary e-book format. If they release a version that also does e-pub, I may reconsider. Until then it’s “Yes” to Amazon tree-books, “No” to Amazon e-books.

Damn it, I knew when I was writing this blog entry that I was going to regret it. Oh well.

Sunday, 1 August 2010

How I Improved My HTC Desire Battery Life

Reset Battery Stats
NB: This requires root access
  1. Charge phone to 100%.
  2. Re-boot into Clockwork Mod recovery.
  3. Select Advanced --> Wipe Battery Stats
  4. Re-boot phone.
  5. Run phone until it runs out of battery.
WTF? Why is "Android System / OS" Eating My Battery??
Occasionally some kind of runaway process appears to take over my phone and make it lose battery power more quickly. It's either Android System, or Android OS which is apparently using the battery. I haven't been able to work out why this is happening, but I have stumbled upon a workaround. Yep, it's the IT Crowd workaround.


With CyanogenMod 6, re-booting is as simple as holding down the power button and selecting 'Reboot'. However, I'd far rather make the technology work for me, rather than make me a slave to the machine. So I installed 2 apps to enable me to automate a reboot every morning:
  1. AppAlarm Pro - there's a free version available, but I'd rather pay to support the developer.
  2. Quick Boot
I set up an alarm in AppAlarm pro which launches a shortcut to Quick Book / Reboot at 04:45 every morning.

Before doing that I had to switch 3G off to get through a full day. Now, even with 3G running, I have over 50% left at the end of the day, with battery drain of somewhere between 2 and 3% per hour on average.

Update 18th August: Just found a post on XDA developers forum about how to improve battery life. I might give this a try to see if it improves my battery life even more (it's already pretty good following the process from above). See http://forum.xda-developers.com/showthread.php?t=755903.

Monday, 17 May 2010

I Know Global Variables are a Bad Idea...

...so why on earth did I use them in Sponge?! Well, there is a reason for that. Inexperience. Sponge is my first attempt at writing an application primarily in Clojure, and as such was a massive learning experience.

One of the design decisions I had to make early on was how to maintain state. I don't ever really need to consider this when programming in Java - I would have instance variables in an object somewhere. But with Clojure there are no objects. So where to put the state?

The decision that I made (and that I have been regretting all day today) was to create global references in the relevant namespaces. This had the advantage that I could easily access the current state from wherever I wanted in the code, and so it was one less level of organisation that I had to deal with. What could possibly go wrong?

Well, a colleague of mine requested the ability to run multiple sessions at once i.e. from the same Sponge GUI, launch different servers listening on different ports. That wasn't possible with a single global state. So, I have spent the whole of today going through the code adding a level of indirection. Instead of a single global state across multiple global variables, I am creating a single 'session' map which holds all the state. This is a hugely painful process that has taught me a valuable lesson:

Global variables in Clojure are as bad as global variables in any other language. Don't use them.

Friday, 7 May 2010

The Joy of Clojure

I've just bought the early access edition of The Joy of Clojure using the checkout code "infoq35", as listed here. I've only read 16 pages and already I love it.

This seems like the ideal next book to read after Programming Clojure. Idiomatic Clojure here I come!

Friday, 30 April 2010

Operation "Bad Apple"

For a long time Apple didn't really impinge on my consciousness. Any time I tried to use a Mac I went away feeling frustrated. A child of the Windows age, I'd grown up to love Linux through Ubuntu, and the Mac just seemed too alien.

That all changed when I saw the video where Steve Jobs introduced the iPhone.

I knew as soon as I saw that video that I wanted an iPhone. I stood in a queue on the day the original was released. And then, when the 3G version came out, I queued again to upgrade to it. The iPhone acted as a gateway drug to the world of Apple. I couldn't bring myself to use Windows on a day to day basis, so went out and bought a Mac Mini. And then, having drunk the kool aid, I went out and bought a Mac Air. I'd turned from sceptic to true believer through the seductive wiles of the iPhone.

And for a while I was happy. I moved all my development at home to my Macs. My Ubuntu machine went into semi-retirement. The whole "Unix with a friendly interface" thing appealed.

But that's all gone now. When Apple sued HTC for patent violations, it confirmed my suspicion that I had backed the wrong team. And so Operation "Bad Apple" began. I am gradually ridding myself of the bad Apple stench. I have ordered a custom build PC on which I intend to install Ubuntu 10.04. I've ordered a Acer 1810TZ for my portable computing needs. And I already have a HTC Desire.

In my brave new world I will do all my development on Ubuntu, I will write apps for Android, and I will never buy another Apple product again. As far as I'm concerned, Apple are no longer a company for developers.

I'm switching sides.

Wednesday, 28 April 2010

Link: Interview with Rich Hickey

 On what prompted him to write Clojure:
One watershed moment for me was when I designed and implemented a new system from scratch in Common Lisp, which was a new language for me at the time, only to have the client require it be rewritten in C++, with which I was quite proficient. The port to C++ took three times as long as the original work, was much more code, and wasn't significantly faster. I knew then that I wanted to work in a Lisp, but realized that it had to be client-compatible. I worked on bridging Common Lisp and Java, while doing functional-style programming for clients in C#, for a couple of years, then bit the bullet and wrote Clojure.
http://www.simple-talk.com/opinion/geek-of-the-week/rich-hickey-geek-of-the-week/

He always seems to have something interesting to say.

Monday, 26 April 2010

How to Manually Build and Configure Msysgit

Up to now I have held off from putting all my code in github for one reason - I couldn't access private repositories from work because of the corporate firewall I am behind. But the introduction of Smart HTTP Support has changed that. Unfortunately I had some problems getting this working on my work PC, so I have documented the steps I had to go through for future reference.

I'm a heavy Cygwin user so I first tried to clone over https using the Cygwin version of Git. After working out that I needed to set a https_proxy environment variable (in the format username:pass@proxy-address:port) I then hit a problem whereby I couldn't specify my github password. Git prompted for a password, but for some reason wasn't accepting the input when I typed it in. Strange.

After pondering this for a while I decided that there is probably a good reason that msysgit is the recommended method for running Git on Windows, so decided to install it. However, our firewall blocks downloading .exe files. I have full admin rights on my PC, but I can't download .exes. It used to be that you could circumvent this check by adding a query string to the end i.e. some.exe?a=0 - but somewhere along the way our clever admins spotted this flaw. Since I couldn't download the .exe installer, I had to find a way to build from scratch.

The first problem was getting hold of the source code. I couldn't use Git to get it, for obvious reasons, so instead I downloaded zip archives from http://repo.or.cz/w/msysgit.git/ and http://repo.or.cz/w/git/mingw/4msysgit.git (see the links in the shortlog section, for example). I unzipped the msysgit code into C:\msysgit and then unzipped the contents of the 4msysgit archive into the C:\msysgit\git directory. Building and installing was then a simple case of opening a cmd window in C:\msysgit and running msys.bat.

Then I had to add the following to my Windows path: c:\msysgit\bin;C:\msysgit\mingw\bin

I also created a quick launch shortcut with the command: /share/msysGit/add-shortcut.tcl QuickLaunch

Running git --version confirmed that git was installed correctly. I was then able to clone my private repository by:
  • Setting global git config http.sslverify = false
  • Setting the https_proxy variable
  • Running: git clone https://username@github.com/me/myrepo.git myrepo
Now that I know I can clone a private repository, and push changes to it, I am happy to use github for all my future development.

Credit Where Credit is Due...
This article proved very useful in working out what I had to do.

Friday, 23 April 2010

Sponge: For When You're Using Soap

The Problem
At my current job we have a suite of applications that all have the same basic architecture:
  • A Java web-start client talking to...
  • A server side deployed in JBoss which talks to...
  • Back-end .NET web-services
The .NET services front the core systems that the Java application connects to. The Java application is provided to customers who want to do business with us.

Both the Java side and the .NET side have a problem when it comes to keeping a development environment up and running. The Java guys don't really want to have to run the .NET web-service on their machines, and the .NET guys don't really want to have to run JBoss. What tends to happen is that the Java team try to connect to a shared .NET service which is either:
  • Borked or...
  • Incompatible with the version of the code they are developing
Cue lots of wasted time for all involved as people stop that they are doing to fix the shared service.

A Workaround
I wanted to find a way to be able to run up a development environment on my PC without having to rely on the back-end .NET service. I did this with a combination of:
I used SoapTrace to record some Soap exchanges, and plugged the responses into some mock web-services in SoapUI. This worked but it was a total pain in the ass to set up. It took ages to record all the Soap exchanges I needed, then I had to fiddle around in SoapUI to generate the responses, then configure SoapUI to return different responses for different types of request.

What I really wanted was a way to be able to record a Soap session and, at the flick of a switch, use the recorded responses to mock the web-service calls in my development environment.

Enter Sponge
And so Sponge was born. Sponge has the following features:
  • It can act as a Soap router, sitting between the Soap client and the Soap service.
  • Soap exchanges can be saved, viewed, edited etc.
  • Entire Soap sessions can be stored and used to mock out web-service calls.
  • The user can choose which response to return for a particular request type at the click of a button.
  • Previously saved Soap requests can be re-sent to the service.
  • It has a plugin mechanism to enable requests or responses to be dynamically transformed before sending / after receiving.
Sponge is the tool that I couldn't find that allowed me to work the way I wanted to work.  On the Java side, once I have a session recorded I no longer need to rely on the .NET web-service. And on the .NET side, given a session they no longer need to run the Java side of the application in order to generate requests.

The Technology
I had 2 goals when writing Sponge:
  1. Create a tool to help me in my day to day development.
  2. Learn Clojure.
The meat of Sponge is implemented in Clojure, with a thin(ish) GUI layer written in Java (using the Netbeans GUI builder).

Screenshot
This screenshot shows the key features:

  1. The top half of the screen shows some Soap exchanges.
  2. The bottom half shows the Soap request (on the left) and response (on the right) for the currently highlighted exchange.
  3. The response is in edit mode. By double clicking on the text panel it is possible to edit the xml.
  4. The 3rd exchange is highlighed in green, and has 'R' in the info column. This indicates that this row will be replayed i.e. if any request comes in that matches the Namespace, URL and Soap method of the replay row, then that response will be returned.
  5. The Replays column shows the number of times that response has been replayed.
Interested?
You can download the latest version of Sponge from http://github.com/NeillAlexander/sponge. It is licensed under the Eclipse Public License, and all the source code is available in Github. I don't intend to do any more development at this point in time. It does what I need it to do.

Installation instructions are included in the distribution, and there is also some basic integrated help which describes how to use it.

Feedback
If you make use of this, let me know what you think. Leave a comment here, or raise an issue on Github, or whatever. I'd love to know that someone else finds this useful.

Sunday, 4 April 2010

Goodbye iPhone, Hello Android

"Whooh! It's weird not smoking, I'll tell you that. But I'm glad I quit y'know because I felt like to be honest with you I was on the wrong side of the war against drugs, because I smoked cigarettes and gave the tobacco lobbyists and the tobacco growers any more fuckin' money for the poison they spread, and advertise all over our world..." (Bill Hicks, Revelations)
When Apple decided to sue HTC, I felt like I was on the wrong side. So this week I bought a HTC Desire.

My iPhone now lives in a drawer.

Thursday, 11 March 2010

Leiningen or Maven?

Originally posted to the Clojure Google Groups list here:

I'll add my 2 cents worth. I've come from a Java development background to Clojure. A couple of years ago our company migrated from Ant to Maven. At first I really really hated it. I found it difficult to make it do what I wanted it to do. Nowadays, though, it is my build tool of choice.

The turning point was when I realized that the key to using Maven is understanding the Maven model - understanding how Maven wants to build. You don't tell Maven what to do. You just put the files in the places Maven expects, and Maven then _knows_ what to do. My rule of thumb is: if I'm spending a lot of time tweaking configuration and overriding defaults in the pom file then I'm probably doing it wrong.

When I started programming in Clojure I immediately looked for a Maven plugin to build Clojure projects. The first project I worked on I did using the plugin. For my next Clojure project I decided to give Leiningen a spin. Initially I was very impressed. It was easy to get a
project up and running, easy to build, easy to run tests, easy to run a swank server etc. For a couple of weeks I was very happy. But.....

I finally hit a point where Leiningen was no longer working for me. My current project has a Swing GUI, built using Matisse (the Netbeans GUI builder). I couldn't find a way to build the Swing portion and the Clojure portion together using Leiningen. I also couldn't find a way to divide my project up into different sub-modules with one build (maybe I'm guilty of trying to use Leiningen like I use Maven). In any case, I ended up converting the project to use Maven.

Maven has a steep learning curve, but once you get it, it's incredibly powerful.
"The philosophy of Tai Chi Chuan is that if one uses hardness to resist violent force, then both sides are certain to be injured at least to some degree. Such injury, according to tai chi theory, is a natural consequence of meeting brute force with brute force. Instead, students are taught not to directly fight or resist an incoming force, but to meet it in softness and follow its motion while remaining in physical contact until the incoming force of attack exhausts itself or can be safely redirected, meeting yang with yin." (http:// en.wikipedia.org/wiki/Tai_chi_chuan)
If you fight with Maven, you will lose. If you meet it in softness and follow its motion you will attain build enlightenment...

;-)

Finally, I agree with Stuart that polyglot Maven could / should be the way forward. I don't really wrestle with pom files much manually anyway since Netbeans does a very good job of automatically updating them. But Leiningen has given me a taste of writing build files in Clojure, so I will definitely be investigating the polyglot option soon.

To learn more about Maven I would recommend:
http://www.sonatype.com/books/mvnref-book/reference/public-book.html and http://www.sonatype.com/books/mvnex-book/reference/public-book.html

Saturday, 20 February 2010

Thursday, 18 February 2010

Object to Classes - Use Maps

In the 10 years that I have worked as a computer programmer I have worked almost exclusively with object oriented languages. Primarily Java, but also Ruby, Groovy, and Perl (though object oriented Perl wasn't much fun!). Thinking in objects therefore comes very naturally to me. Part of the challenge of learning Clojure is that it isn't an object oriented language. I'm having to re-wire my brain to work in the functional paradigm, something that is both a challenge and great fun.

I read about Abstraction Barriers in SICP and wanted to apply it to the tool I'm currently developing in Clojure. I created a new namespace to encapsulate the concept of a 'version' (as in a software version - I need functions to manipulate version strings like 1.0.1-b01-SNAPSHOT). Based on what I had read in SICP I created functions like 'make-version' to act as the abstraction barrier. But then the question arose - what should this actually return? Initially, again inspired by SICP, make-version created a closure with a message-passing dispatch function:
(defn make-version [major minor patch build snapshot]
  (fn [selector]
    (cond
     (= :major selector) major
     (= :minor selector) minor
     (= :patch selector) patch
     true (throw (IllegalArgumentException. "Selector not recognized"))))
  )
But it occurred to me that by doing this I would lose all the benefits of the data structures Clojure provides wrt concurrency etc. So I thought about the implementation a bit more, and did some reading on the Clojure Google group. After doing a search for "data abstraction" on the group I found a post by Rich Hickey that lit a light-bulb in my head:
 "I know people usually think of collections when they see vector/map/ set, and they think classes and types define something else. However, the vast majority of class and type instances in various languages are actually maps, and what the class/type defines is a specification of what should be in the map. Many of the languages don't expose the instances as maps as such and in failing to do so greatly deprive the users of the language from writing generic interoperable code. 
Classes and types usually create desert islands. When you say:
//Java 
class Foo {int x; int y; int z;} 

--Haskell 
Foo = Foo {x :: int, y :: int, z :: int}
you end up with types with a dearth of functionality. Sure, you might get hashCode and equals for free, or some other free stuff by deriving from Eq or Show, but the bottom line is you are basically starting from scratch every time. No existing user code can do anything useful with your instances."
[...snip...]
"I guess I want to advocate - don't merely replicate the things with which you are familiar. Try to do things in the Clojure way. If your logical structure is a mapping of names to values, please use a map. Positional data is fragile, non-self-descriptive and unmanageable after a certain length - look at function argument lists. Note that using maps doesn't preclude also having positional constructors, nor does it dictate a space cost for repeating key names - e.g. structmaps provide positional constructors and shared key storage. "
As a follow up, Stuart Sierra posted a link to a blog entry he had written about how to model data. In it he directly contrasts the difference between the OO mindset and the Clojure / functional mindset:
"So here’s a slightly radical notion: don’t use classes to model the real world. Treat data as data. Every modern programming language has at least a few built-in data structures that usually provide all the semantics you need.
It all boils down to this:
"It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures."(Alan Perlis)
In OO programming we create data structures at the drop of a hat, but that's not the Clojure way. The Clojure way is to use the data structures provided by the language, and the vast library of functions that know how to manipulate them.

So yes, it's fine to have an abstraction barrier with functions like make-version. But there is no need to create custom data structures using closures, message-passing, and dispatch functions. Just use a map. It's simple, it works, and it's the Clojure way.

Wednesday, 17 February 2010

mvn clojure:swank throws java.lang.NumberFormatException: Invalid number: 2009-09-14

For some reason the 'swank' target of the clojure-maven-plugin isn't working for me on Windows. At home, on Ubuntu, it works fine. At work, with the same code base, it doesn't work. It throws the following exception:

Exception in thread "main" clojure.lang.LispReader$ReaderException: java.lang.NumberFormatException: Invalid number: 2009-09-14

I decided to work around this by extending my 'clj' starter script. I added a --pom switch which tells the script to build the classpath using the pom file in the current directory. It makes use of the build-classpath target of the maven-dependency-plugin. So now at work I can start up a Swank server simply by typing:

clj --pom "c:\dev\nwalex.com\clojure-scripts\start-swank.clj"

I have this aliased to 'swank' in my .aliases file in Cygwin.

Anyway, here's the full script:
#!/bin/bash

function init_classpath {
    # set up the classpath dynamically. Note this includes the jline jar
    CLASSPATH=""
    for jarfile in `ls -l ~/.clojure-classpath/ | pcol 11`; do
        JAR=`cygpath --windows $jarfile`
        CLASSPATH="$CLASSPATH;$JAR"
    done
}

function init_classpath_from_pom {
    echo "Initializing classpath from pom.xml..."

    # create the classpath file
    mvn dependency:build-classpath -Dmdep.outputFile=classpath 2>&1 > /dev/null

    # store it and remove the file
    CLASSPATH=`more classpath`
    rm classpath

    # also add everything under the clojure source directories
    CLASSPATH="$CLASSPATH;./src/main/clojure;./src/test/clojure"
}
 
if [ $# -eq 0 ] ; then
    init_classpath
    stty -icanon min 1 -echo
    java -Djline.terminal=jline.UnixTerminal -cp $CLASSPATH jline.ConsoleRunner clojure.main
else
    TMPFILE=""
    while [ $# -gt 0 ] ; do
        case "$1" in
        --pom)
            init_classpath_from_pom
            ;;
        -cp|--classpath)
            CLASSPATH="$CLASSPATH;$2"
            shift
            ;;
        -e)
            TMPFILE="/tmp/$(basename $0).$$.tmp"
            /bin/echo $2 > $TMPFILE
            ARGS=$TMPFILE
            break
            ;;
        *)
            ARGS="$ARGS $1"
            ;;
        esac
        shift
    done

    if [ "$CLASSPATH" == "" ] ; then
        init_classpath
    fi

    if [ "$ARGS" != "" ]; then 
        ARGS=`cygpath --windows $ARGS`
    fi
 
    java -cp "$CLASSPATH" clojure.main $ARGS
    if [ "$TMPFILE" != "" ] ; then
        rm $TMPFILE
    fi
fi
start-swank.clj is simply:
(require 'swank.swank)
(swank.swank/start-server "nul" :encoding "utf-8" :port 4005)
Update 1: Updated script to include the src/main/clojure and src/test/clojure directories on the classpath.

Sunday, 24 January 2010

Speaking in Sentences

I have done quite a lot of programming in Clojure over the last couple of days, and have finally moved past the initial, hesitant stage where I would sit for ages trying to work out what I needed to type. I'm no longer looking up the dictionary to translate individual words. Now I'm speaking in sentences.

This is a very important point to reach because it means I have moved beyond the initial frustrating stage where, sat with my fingers on the keyboard in front of Emacs, connected to a repl, I just didn't know what to type. Every little step forward was painful. As the code flows more and more easily from my fingertips, so it becomes more and more satisfying.

Today, for example, I was trying to convert a series of 163 mp3 files into an audiobook on iTunes, consisting of 15 chapters. Since I couldn't find any nice friendly tools on the Mac, I switched over to my Ubuntu PC to do some proper work. Previously I would have written a little Perl script to automate the tasks I wanted to do, but this time I decided to do it in Clojure. And what a joyful experience that turned out to be.

Let me give you an example. I had already, a few years ago, sorted the 163 mp3 files into 15 different folders. The files are an audio recording of a General Semantics seminar given by Alfred Korzybski himself in 1949. Each folder contained the files for individual lectures. The first thing I wanted to do was tidy up the file names to remove spaces. So I kicked off a repl and started to write some code in Emacs.

First I got a handle on all the mp3 files:
(defn has-ext? [ext file]
  "Does the file end with the extension ext"
  (.endsWith (.getName file) (str "." ext)))

(def mp3? (partial has-ext? "mp3"))

(defn mp3s
  [dir]
  (filter mp3? (file-seq (in-dir dir))))
At the repl I was then able to do something like:
(def mp3files (mp3s "/path/to/top/level"))
mp3files then contains a sequence of java.io.File objects for each of the files. So now I can use this to rename the files:
(defn replace-str
  "Wraps STring replaceAll"
  [pattern in with]
  (.replaceAll in pattern with)
  )

(defn replace-whitespace
  "Replace whitespace in 'in' with 'with'"
  [in with]
  (replace-str " " in with)
  )

(defn rename-file
  "Rename the file using the single arg fn to transform the file name"
  [file fn]
  (.renameTo file (java.io.File. (.getParentFile file) (fn (.getName file)))))

(defn rename-files
  "Rename all the files in the fseq using fn to transform the name"
  [fseq fn]
  (map #(rename-file %1 fn) fseq)
  )
With these functions, replacing the whitespace was as simple as typing the following in the repl:
com.nwalex.gs> (rename-files mp3files #(replace-whitespace %1 "-"))
This kind of dynamic programming is so much more satisfying than trying to write a Perl script that will finally (hopefully) work only once all the code has been written. Clojure lends itself to writing a little bit of code at a time, sketching out your solution and evolving it. Once I had the handle to the java.io.Files, I could play around with them, experiment with how to extract the names etc. It's just an incredibly satisfying way to work.

I'm now beginning to think in Clojure. The next stage is to gain more familiarity with the core and contrib apis. I'm speaking in sentences, now I need to improve on the vocabulary.

Saturday, 23 January 2010

"Seeing" by Jose Saramago


I finished reading "Seeing" by Jose Saramago this morning. What a wonderful book. It is a sequel of sorts to "Blindness", though I didn't realize this until halfway through.

Words to describe this book: funny, intelligent, gripping, angry, sad. The phrase 'biting satire' is too mild. This doesn't so much bite as dismember with ruthless precision. The world Saramago describes doesn't seem too far removed from our own.

I went straight from the coffee shop where I devoured the last 30 odd pages to Waterstones to find more by the same author. I left with 2 in my hand, in eager anticipation of what he will serve up next.

Abstracting

In this blog entry I want to demonstrate the elegance of Clojure by showing how a function I wrote evolved. This is for a little tool that I have written previously in Perl and in Groovy, that I'm now writing it in Clojure. The aim of the function is to recursively find all the pom files in a specific directory.

Here is version 1:
(defn pom-files-under
  "Find all the pom files in the current directory"
  [dir]
  (filter pom-file? (file-seq (File. dir)))
  )

(defn pom-file?
  [file]
  (and (= "pom.xml" (.getName file))
       (not-under-target? file)))

(defn not-under-target?
  [file]
  (not (.contains (.getAbsolutePath file) "target")))
This worked, but I wasn't particularly happy with it. There is no checking that the parameter to pom-files-under is actually a directory and the pom-file? method is ugly. So I scrapped that code and started again. Here is version 2:
(defmulti in-dir
  "Abstracts over the concept of a directory, always returing a java.io.File
that is guaranteed to be a directory"
  class)

(defmethod in-dir String [s]
  (in-dir (java.io.File. s)))

(defmethod in-dir java.io.File [f]
  (if (.isDirectory f)
    f
    (throw (IllegalArgumentException. "File is not a directory"))))

(defn has-name? [name file]
  (= name (.getName file))
  )

;; partial application of has-name? checking if file name is pom.xml
(def pom? (partial has-name? "pom.xml"))

(defn not-under-target?
  [file]
  (not (.contains (.getAbsolutePath file) "target")))

(defn find-files
  "Find files in directory that match predicates pred & others"
  [in-dir pred & others]
  (filter pred (file-seq in-dir))
  )

(defn pom-files
  "Find all pom files recursively in directory in-dir"
  [dir]
  (find-files (in-dir dir) pom?)
  )
This seems much more elegant.The in-dir multi-method now guarantees to return a java.io.File object that represents a directory. And the find-files method now takes in multiple predicates, the idea being that you supply the directory and the predicates, and it returns the files that match all predicates. The only problem is that the find-files method doesn't actually work at this point. I couldn't for the life of me work out how to implement that functionality. And so I posted a plea for help on the Clojure Google Groups board. The advice I got back really opened my eyes to the kind of abstraction possible in Clojure.

The first solution I implemented based on this advice was:
(defn find-files
  "Find files in directory that match predicates pred & others"
  [in-dir pred & others]
  (reduce (fn [xs f] (filter f xs)) (file-seq in-dir) (cons pred others)))
This fixed the find-files function, making it apply all predicates. Then a discussion started about how to abstract this out further, leading to the following comment from Perry Trolard:
I think it's easier to think about combining predicates separately from your file-filtering code.
Then Sean Devlin followed up with this code to combine predicates:
(defn every-pred?
    "Mimics AND"
    [& preds]
    (fn [& args] (every? #(apply % args) preds)))

(defn any-pred?
    "Mimics OR"
    [& preds]
    (fn [& args] (some #(apply % args) preds))) 
I incorporated this into my code, and did a bit more abstracting, leading to this final version (the rest of the code remained the same):
(def target? (partial has-name? "target"))

(defn not-under-target?
  [file]
  (not (target? (.getParentFile file))))

(defn every-pred?
    "Mimics AND"
    [& preds]
    (fn [& args] (every? #(apply % args) preds))) 

(defn pom-files
  "Find all pom files recursively in directory in-dir"
  [dir]
  (filter (every-pred? pom? not-under-target?) (file-seq (in-dir dir))))
This process of abstracting really opened my eyes to what is possible in Clojure. I love the elegance and expressiveness possible in this language. The only problem is, the more I program in it, the less I want to go back to my day job of programming in Java! It seems so clunky and primitive now!

Friday, 22 January 2010

Setting up the Clojure Classpath for Utility Scripts in Cygwin

I'm beginning to really enjoy programming in Clojure, but it's still a struggle. The best way to improve is to use it every day. And so I decided to configure my environment to make it easy to write scripts in Clojure. Over the last few years, whenever I have learned a new language, I have used it to write little utility scripts for day to day tasks at work. I have scripts written in Bash, Perl, Groovy, and JRuby. I decided it was time to add Clojure to the list.

I use Cygwin at work, so my starting point was the clj Bash script at http://en.wikibooks.org/wiki/Clojure_Programming/Getting_Started#Create_clj_Script. I used this one:
CLASSPATH="/path/to/clojure.jar"
if [ $# -eq 0 ] ; then
    JLINE="/path/to/jline.jar"
    CLASSPATH=$JLINE:$CLASSPATH
    java -cp $CLASSPATH jline.ConsoleRunner clojure.main --repl
else
    TMPFILE=""
    while [ $# -gt 0 ] ; do
        case "$1" in
        -cp|-classpath)
            CLASSPATH=$CLASSPATH:$2
            shift
            ;;
        -e)
            TMPFILE="/tmp/$(basename $0).$$.tmp"
            /bin/echo $2 > $TMPFILE
            ARGS=$TMPFILE
            break
            ;;
        *)
            ARGS="$ARGS $1"
            ;;
        esac
        shift
    done
 
    java -cp $CLASSPATH clojure.main $ARGS
    if [ "$TMPFILE" != "" ] ; then
        rm $TMPFILE
    fi
fi
I had to edit it a bit to make it Windows friendly, but the script basically worked. However, I wasn't satisfied. I didn't want to have to manually edit the script, or an environment variable, simply to add jar files to the classpath. I wanted a simpler, more obvious way. And so I created a directory in my home directory called .clojure-classpath. The contents of the directory are:
$ ls -l
total 24
-rw-r--r-- 1 alexanc mkgroup-l-d 82 Jan 22 11:16 README
lrwxrwxrwx 1 alexanc mkgroup-l-d 92 Jan 22 12:06 clojure-contrib.jar -> /cygdrive/c/m2repository/org/clojure/clojure-contrib/1.1.0-RC3/clojure-contrib-1.1.0-RC3.jar
lrwxrwxrwx 1 alexanc mkgroup-l-d 68 Jan 22 12:06 clojure.jar -> /cygdrive/c/m2repository/org/clojure/clojure/1.1.0/clojure-1.1.0.jar
lrwxrwxrwx 1 alexanc mkgroup-l-d 79 Jan 22 12:42 commons-exec-1.0.1.jar -> /cygdrive/c/m2repository/commons-exec/commons-exec/1.0.1/commons-exec-1.0.1.jar
lrwxrwxrwx 1 alexanc mkgroup-l-d 60 Jan 22 12:06 jline.jar -> /cygdrive/c/m2repository/jline/jline/0.9.94/jline-0.9.94.jar
lrwxrwxrwx 1 alexanc mkgroup-l-d 82 Jan 22 12:08 swank-clojure.jar -> /cygdrive/c/m2repository/swank-clojure/swank-clojure/1.1.0/swank-clojure-1.1.0.jar
I already have a mountain of jar files in my local maven repository so, in order to make them available to Clojure, I simply created symbollic links to them. Then, I modified the clj
Bash script to automajically add all the jar files in this directory to the classpath prior to starting up Clojure. My new, modified version of clj is as follows:
#!/bin/bash
 
# set up the classpath dynamically. Note this includes the jline jar
CLASSPATH=""
for jarfile in `ls -l ~/.clojure-classpath/ | pcol 11`; do
    JAR=`cygpath --windows $jarfile`
    CLASSPATH="$CLASSPATH;$JAR"
done

if [ $# -eq 0 ] ; then
    stty -icanon min 1 -echo
    java -Djline.terminal=jline.UnixTerminal -cp $CLASSPATH jline.ConsoleRunner clojure.main
else
    TMPFILE=""
    while [ $# -gt 0 ] ; do
        case "$1" in
        -cp|-classpath)
            CLASSPATH="$CLASSPATH;$2"
            shift
            ;;
        -e)
            TMPFILE="/tmp/$(basename $0).$$.tmp"
            /bin/echo $2 > $TMPFILE
            ARGS=$TMPFILE
            break
            ;;
        *)
            ARGS="$ARGS $1"
            ;;
        esac
        shift
    done
 
    java -cp $CLASSPATH clojure.main $ARGS
    if [ "$TMPFILE" != "" ] ; then
        rm $TMPFILE
    fi
fi
The only non-standard thing in the script is the call to pcol which is a little Perl script I wrote to split a line on whitespace and print the column specified by the number. This modified version of clj:
  1. Lists all the files in the .clojure-classpath directory, and gets the full path from the output of ls.
  2. Converts the path to a Windows friendly path.
  3. Dynamically adds every file to the classpath.
  4. Starts Clojure.
That worked beautifully. Now I can run stand-alone scripts by simply typing: clj [script_name].

The next problem I had to solve was how to edit my scripts in Emacs using the same classpath. Fortunately I knew that I could do this by starting an instance of a Swank-clojure server from within the repl on the command line. Note that the swank-clojure.jar is in the .clojure-classpath directory. The script to start the server is simply:
(require 'swank.swank)
(swank.swank/start-server "nul" :encoding "utf-8" :port 4005)
I can start the server by:
  1. Running: clj start-swank.clj
  2. From a running repl loading start-swank.clj.
  3. Entering the commands directly into the running repl:
user=> (require 'swank.swank)
nil
user=> (swank.swank/start-server "nul" :encoding "utf-8" :port 4005)
Connection opened on local port  4005
#
Bingo. I can now edit my scripts in Emacs using Slime, with exactly the same classpath that will be used to run them. And since the .clojure-classpath directory contains a symbollic link to the directory I keep the scripts in (not shown in the listing above - I copied that before I added the link to the directory), all the scripts I write will automatically be on the classpath, and hence I can start building up a library of utility functions.

I am delighted with this set up, and can't wait to start writing some utility scripts in Clojure using Emacs.

Monday, 11 January 2010

Documenting Clojure Code

As I begin to write Clojure code, I want to make sure I document it correctly. Looking at some source I can see the following (from http://github.com/richhickey/clojure-contrib/blob/master/src/clojure/contrib/core.clj):
(ns
  #^{:author "Laurent Petit (and others)"
     :doc "Functions/macros variants of the ones that can be found in clojure.core 
 (note to other contrib members: feel free to add to this lib)"}
  clojure.contrib.core
  (:use clojure.contrib.def))

(defn new-by-name
  "Constructs a Java object whose class is specified by a String."
  [class-name & args]
  (clojure.lang.Reflector/invokeConstructor
   (clojure.lang.RT/classForName class-name)
   (into-array Object args)))
There is some meta data in the namespace declaration, and a documentation string in the function definition. This is what I should look to emulate. ":doc" must be a standard meta-data label. Yep - page 57 of Programming Clojure confirms it. Others include:
  • :arglists - Parameter info used by doc
  • :doc - Documentation used by doc
  • :file - Source file
  • :line - Source line number
  • :macro - True for macros
  • :name - Local name
  • :ns - Namespace
  • :tag - Expected argument or return type

Sunday, 10 January 2010

Configuring My Development Environment

I'm going for the following configuration:
  1. Use NetBeans to do all the Java / Maven specific stuff.
  2. Use Emacs to do all the Clojure programming.
  3. Use the clojure-maven-plugin to compile the .clj files, and to start the swank server for the project.
If I understand it correctly, this means that I can add dependencies to my pom file, then start up the swank server using "mvn clojure:swank". That will make all the jar files pulled in by Maven available in the Slime repl in Emacs.

I could do all my development in Emacs, but I am unwilling to part with NetBeans. For the pure Java / Maven side, I am very comfortable with NetBeans. This setup gives me the best of both worlds - NetBeans does what it is good at. Emacs does what it is good at.

Let's test this out. I will create a project, add a Jakarta Commons dependency, then try to call it from the repl. Here is the pom:
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemalocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelversion>4.0.0</modelversion>
    <groupid>com.nwalex</groupid>
    <artifactid>mrtj-clojure</artifactid>
    <packaging>jar</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>mrtj-clojure</name>
    <url>http://maven.apache.org</url>

    <repositories>
        <repository>
            <id>build.clojure</id>
            <url>http://build.clojure.org/snapshots</url>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupid>org.clojure</groupid>
            <artifactid>clojure</artifactid>
            <version>1.1.0</version>
        </dependency>
        
        <dependency>
            <groupid>com.codestuffs.clojure</groupid>
            <artifactid>swank-clojure</artifactid>
            <version>1.1.0</version>
        </dependency>
        <dependency>
            <groupid>commons-lang</groupid>
            <artifactid>commons-lang</artifactid>
            <version>2.4</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            
            <plugin>
                <groupid>com.theoryinpractise</groupid>
                <artifactid>clojure-maven-plugin</artifactid>
                <version>1.3.1</version>
                <executions>
                    <execution>
                        <id>compile-clojure</id>
                        <phase>compile</phase>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>
</pre>
Some things to note:
  1. I added a repository to pick up the clojure dependency
  2. I had to manually install the swank-clojure dependency to my local repository.
If I then start up the swank server with "mvn clojure:swank", I am able to connect to it from Emacs using M-x slime-connect. Now, am I able to access the Jakarta Commons Lang classes?
user> (org.apache.commons.lang.StringUtils/isBlank " ")
true
user> (org.apache.commons.lang.StringUtils/isBlank "not blank ")
false
Yes! That proves the set up.

And does the clojure-maven-plugin compile my .clj files?
neill@korzybski:~/src/mrtj-clojure$ ls target/classes/com/nwalex/mrtj
test$hello_world__3.class  test__init.class  test$loading__6309__auto____1.class
Yes it does.

I am very happy with this. I started down this road yesterday afternoon when I struggled and failed to configure the NetBeans Enclojure plugin to work with Clojure 1.1.0. 1 day later I have a working Clojure development environment integrating NetBeans, Maven, and Emacs. I don't need to mess about with classpaths. Maven will sort all of that out for me when I add dependencies to the pom.

Time to start hacking!

Further Adventures in Emacs

I've decided, if I want to be a Clojure programmer - if I want to be a serious hacker - then I need to learn Emacs. Enough screwing around. I don't need to learn it all at once. If I start using it now, and learn the shortcuts as and when I need them, then in a few months (years?) I should be fairly proficient.
So - I need to set Emacs up on my main Ubuntu PC, on my Mac, and on my Windows work PC. I already tried to set up my work PC using the Emacs Starter Kit, but I couldn't make it work with my company's proxy. I therefore plan to set it up on my Windows PC at home, then send myself a tar ball of the .emacs.d directory for work.

But, first things first. I need to install Emacs + Clojure + Slime on my Ubuntu PC. In a previous post I linked to some articles which described how to set this up. However, I have since discovered that the preferred way is to use the Emacs Starter Kit to install swank-clojure. So that's what I'm going to do. I will document the steps for future reference, and for anyone else who wants to do the same.

From home directory:
  1. Install Emacs: sudo apt-get install emacs23
  2. Get the Emacs Starter Kit: git clone git://github.com/technomancy/emacs-starter-kit.git .emacs.d
  3. Run Emacs: emacs &
  4. In Emacs, install swank-clojure: M-x package-list-packages, put an 'I' beside package swank-clojure, and then press 'x' to install.
  5. Run slime: M-x slime
  6. That should prompt to install Clojure. Type 'y' and all should be working.
When I first tried to install this I was getting problems with starting up Slime.
user=> java.io.FileNotFoundException: Could not locate swank/swank__init.class or swank/swank.clj on classpath:  (NO_SOURCE_FILE:0)
user=> user=> java.lang.ClassNotFoundException: swank.swank (NO_SOURCE_FILE:0)
user=> user=> nil
java.lang.ClassNotFoundException: swank.swank (NO_SOURCE_FILE:0)
user=> user=>
It turns out this was because I had already manually installed Clojure into my ~/.clojure directory. When I removed the .clojure directory and tried again, it all worked fine. Watching the video at http://www.bestinclass.dk/index.php/2009/12/clojure-101-getting-clojure-slime-installed/ pointed me in the right direction for solving this. I realized that I wasn't being prompted to install Clojure, which suggested to me that I should mask the manually installed version.

Now to try to do the same on Windows...

Update 1: I followed the same process on my Windows PC, but got the following error when installing swank-clojure: "Local variables entry is missing the suffix". A quick Google brought me to http://groups.google.com/group/clojure/browse_thread/thread/c4d00ba0f1614c49?pli=1. Basically the solution was to edit my ~/.emacs.d/package.el file to make the changes indicated in that post. I've placed the edited version that I used at: http://files.nwalex.com/package.el. Use at your own risk.

Update 2: I also made the following changes to make Emacs work better on my work PC. The Emacs Starter Kit will look for a file called [username].el, and load it as part of the initialization process. So I created a file called alexanc.el in .emacs.d and added the following to make it work well with our proxy:
;;; set up the proxy
(setq url-using-proxy t)
(setq url-proxy-services
'(("http" . "our-proxy:8080")))
I'm starting to really enjoy working with Emacs.

Saturday, 9 January 2010

Exporting Code Templates from NetBeans

After upgrading to NetBeans 6.8 I realized I no longer had the custom code templates I had put together. Although NetBeans has functionality to export settings, the code templates are not included. There is an issue raised in Mozilla about this: https://netbeans.org/bugzilla/show_bug.cgi?id=99494.

Fortunately there is a workaround, as described in this comment:
In fact there is a workaround for this, the codetemplates are stored in an xml file in your userdir - eg.[userdir]/config/Editors/text/x-java/abbreviations.xml. You can copy this file over to your mates' userdir and they should see the same codetemplates in their IDEs.
This worked for me.

Using iPhone Calendar and Remember the Milk

In a previous post I described my Remember the Milk set up. As part of my switch to RTM I moved all the entries from the Calendar app on my iPhone into RTM. This worked very well. The one problem I had was that there was no way to view future tasks easily. I wanted to be able to have a calendar view of upcoming tasks, something I could glance at quickly to determine if I was free at some future date.

The solution for this problem was to use the iCalendar feeds RTM provides to add certains tasks to Google Calendar. Then using Google Sync it was possible to synchronize Google Calendar with the calendar on my iPhone. The net result is that anything tagged with 'appointment' in RTM now automatically appears in my calendar. I don't enter anything directly into the calendar. I simply use the Calendar app on my phone as the calendar view the RTM app lacks.

The one problem I had setting this up was while adding the calendar to Google Calendar. For some reason it just wouldn't import. Other people have had this issue too. It took about 6 attempts to add the iCalendar feed to Google Calendar.

Friday, 1 January 2010

ACL: Chapter 3

Well, the exercises at the end of chapter 3 really forced me to think. I had to really bend my brain to force it to think in functional, rather than imperative terms.

I have decided to try to provide answers to the exercises in the best idiomatic Clojure code that I can manage, including using the clojure-contrib library where possible. I think this is the best way for me to learn to think in Clojure.

Exercise 3 wasn't too bad. It just required a little bit of lateral thinking to implement the required functionality.
; exercise 3
; write a version of union that preserves the order
; of the original lists
(defn new-union [lst1 lst2]
  (distinct (interleave lst1 lst2)))
Exercise 4 really stumped me for a while. It was while doing this exercise that I realized how much work I had to do to think functionally, rather than imperatively. In the end I searched through the api docs to find something that implemented this functionality. I found clojure.contrib.seq-utils/frequencies and looked at the source code, which led me to the reduce function, and hence to the following solution:
; exercise 4
; define a function that takes a list
; and returns a list indicating the number
; of times each element appears, sorted
; from most common element to least
; common
(defn occurrences [lst]
  (sort-by val > (reduce 
                   (fn [counts x]
                     (assoc counts x (inc (get counts x 0))))
                   {}
                   lst)))
In my drive to attempt to write concise, idiomatic Clojure code, making use of libraries, I then re-wrote the function as:
; better version using clojure.contrib.seq-utils/frequencies
(defn better-occurences [lst]
  (sort-by val > (clojure.contrib.seq-utils/frequencies lst)))
That was blood worth sweating. It forced me to hunt through the api documentation, and taught me how to use reduce.

By the time I reached exercise 5 I was beginning to enjoy the power of the sequence library.
; exercise 5
; Suppose the function pos+ takes a list and returns a
; list of each element plus its position
; (pos+ '(7 5 1 4))
; (7 6 3 7)
; Define this function using a) recursion, b) iteration
; c) mapcar (not sure this exists in clojure)

; a) recursion
(defn recursive-pos+ 
  ([lst] (recursive-pos+ lst 0))
  ([lst pos]
    (if (empty? lst)
    ()
    (cons (+ pos (first lst)) (recursive-pos+ (rest lst) (inc pos))))))


; b) iteration
(defn iterative-pos+ [lst]
  (loop [result-list () working-list lst pos 0]
    (if (empty? working-list)
      (reverse result-list) ; yuck
      (recur (cons (+ pos (first working-list)) result-list)
        (rest working-list)
        (inc pos)))))


; c) with map - seems to do what mapcar would do
(defn map-pos+ [lst]
  (map (fn [x pos] (+ x pos)) lst (iterate inc 0)))
I just love the elegance of map-pos+.

After all that, exercise 8 came quite easily:
; exercise 8
; define a function that takes a list and prints it
; in dot notation
; (showdots '(a b c))
; (A . (B . (C . NIL)))
(defn showdots [lst]
  (if (empty? lst)
    (str "NIL")
    (str "(" (first lst) " . " (showdots (rest lst)) ")"))) 
As I said above, I've begun to appreciate the difference between thinking in imperative and functional terms. Working through these exercises I feel I am beginning to grok the functional approach. I have a long way to go, but the journey is proving fun.