Clojure Cup — experience participating in a hackathon on writing web applications in Clojure

This weekend went Clojure Cup — 48 hour hackathon in Clojure. It will be a relatively long post about how he passed in our team, with technical details and other things. Well, in the end of the post so far without it, will tell quite a bit about our project. And those who are too lazy to read, you can go directly to CodeNotes and even vote for us on the Clojure Cup.

Working environment

In participating in the hackathon, there are two different "modes": the participation for fun and to win. In the first case we lazily code something cool for two days; in the second it's a little different, hardcore and early start of the preparation. We chose the second option.
We had a team of four — two, including me and si14, located in St. Petersburg, one — in Tyumen (@zloy_alu), and one in Munich. But no problem in communication we have not experienced — helped more or less successful insulation of tasks, great service Trello simply divine FlowDock. The latter, in my opinion is an ideal service for team collaboration. Judge for yourself — there is a chat and file sharing, and private discussion threads, and notifications about various events (deploy, push to the repository, a new task in trello, a mention on Twitter). It all looks like this:


Since we were definitely planning to win, preparing we began to advance. In particular:
the

the
• checked the selected library and the structure of the application as a whole (see below for more information about Clony);
the
• checked the integration with third party services like NewRelic;
the
• started a blog and Twitter;
the
• learn how to quickly and easily deplot (Makefiles and Ansible);
the
• prepared the documentation for the JSON API between the server and web app om that allowed us to write these pieces more or less independently;
the
• prepared sketches of the interface on paper;
the
• bought the domain and write the SSL certificate.
the
• learn how to configure nginx for normal operation with websockets and static resources;
the
• have prepared a contact pool of beta testers (here came some fail — we were delayed and sent only 11 Moscow, so I managed to only get one review before the end of the hackathon).

Technical details

And now the details that may be useful for web developers on Clojure.

Very useful was the experience of creating a small application Clony. This is an extremely simple thing in Clojure using http-kit (http server and client for Clojure) and Compojure (router) that communicates with its frontend by websockets. It uses Stuart Sierra's workflow, so the development becomes significantly easier: we completely reset our entire application (or its pieces) from nrepl in a split second. During the hackathon, we made a few improvements that are waiting backport'and Clony.

the first Day

We knew in advance that will be deployed on a virtual machine, so learn to use Ansible. Thanks to him we were online after 4 hours from the start of the competition — with a fully deployed production-configuration and test application. It would have happened sooner, if not for the problem with DigitalOcean, to provide hosting for parties (they banned our account because of suspected fraud). We also had the local path (Vagrant — super!) with completely identical configuration, but in the end, all scored on them and deplore once in production. It happened including the fact that some things had to change on the fly on the server hands — from the configuration of MariaDB (damn these ANSI_QUOTES! It was better to take Postgres) and ending with the fact that we forgot to write the logs to disk. Plus we did a fairly complete integration with NewRelic. Can't say that it helped us critically, but the ability to easily see so a large number of parameters — the server response time, rendering time Dom'e and clients, network latency, amount of free memory on the server JVM GC statistics — seemed to soothe his nerves. Library for integration was very small and we put in open-source.
The first day we spent a lot of time trying to make clang — integration Angular.js with ClojureScript. The abundance of small troubles eventually forced to abandon ClojureScript entirely and switch to bare JavaScript. ClojureScript is much better as a language, but in terms of the hackathon to play with your lack of understanding of its subtleties was simply no time.

Trying to make it more correct, we decided to add in the project library for SQL migrations. It was interesting to discover that a very popular library for migrations Lobos not working at all. As a result, we lost about an hour of searching and trial new; stopped Ragtime.

As a result, the end of the first day we had a beautiful home and polarboy list of repositories (a short description of our project below). In the interval 22:00-03:00 team wandered off to sleep (by this time nobody slept with 4 am) and again we gathered around 8 am on Sunday.

Day two

Sunday things got better, but the problems were many:

the

the
• had to redo a small piece of OAuth-authorization;
the
• at some point we banned GitHub over the limit of API requests, which was extremely frustrating and we had to reduce the number of sent queries;
the
• to generate the images we first tried to use something in pure Java, but in the end took a binding to ImageMagick. There are two: one is working through sh and pipes, the other using JNI. The first slow (30+MS for a picture), take a second and failed;
the
• and slow generation of the images and with the number of requests to the GitHub help us out core.cache — great "core" library Clojure;

we decided to take core.async for events on the server. In the end, it was a good decision: the first working version of the code using it was born an hour and a half and it almost was not, despite the fact that no one in the team he isn't used. Using core.async we've made the ability from any code whether to write to the websocket user-specific and complex logic restart scanning of repositories in case of overload the server: we process the request queue to the scanning, rejecting everything that overflows the queue, and in the absence of jobs in the primary queue select from the database that was dumped when overloaded. The code is very compact and seems to work; the

• bugs in the client, some of which, alas, we don't have time to fix it before the end of the hackathon.

Overall, the second day was more than productive. We tried to focus on the first impression, so I paid attention to the little things: a redirect to our own domain, beautiful "How to Use" clear text on the front, nice design, die with "sorry, you don't have a TODO in the repository, here is the bug report form if it is not." Of course, to accelerate the development, we used Bootstrap, but tried to make everything as nice and "nedovoljno". How this all helps us to get the best scores from judges, we learn on Friday.

Most interesting — what's it all about?

Will be strange to not say anything at all about our project.

It seems to us, humanity has not yet invented the perfect bug tracker. However... Why do we have a bugtracker at all? The essence of our project is to remove a separate bug from your life. And we have tried and will continue to try so that he could look like this:

def somefun()
# TODO: @si14, please fix this before release
return true

And we will do the rest — will send the user a notification that you have created for him a task, let's display it in our interface, and will follow your commits and scan the code on the subject of new TODO, FIXME and NOTE. In addition, you can skanirovat beautiful picture for your repository on GitHub showing how TODO in your code and a link to information about it. Our project CodeNotes yet does not possess all desirable functional (plus there while are a little enough annoying bugs what to hide), but when the voting ends, we will continue to improve it and make it the best bug tracker for small projects.
And yet, by the way, you can vote for us and your comments about what you think about the idea of hackathons, and generally about web programming in Clojure.

Article based on information from habrahabr.ru