Subtitel

A blog by Juri Urbainczyk | Juri on Google+ | Juri on Twitter | Juri on Xing

Sunday, November 22, 2015

node.js and MIDI

jazz-midi (www.thejazzpage.de) is a node.js module which allows access to MIDI interfaces. Midi (musical instrument digital interface) is an international standard for communication between electronical music devices which was invented in the early 80ies. Recently, I have been playing around with jazz-midi quite a bit and I figured I should share my experiences here. Now, first of all opening MIDI devices goes like a breeze:

var outName = MIDI.MidiOutOpen(2);
if (outName) {
  console.log('Opened MIDI-Out port: ', outName); 
} else {
  console.log('Cannot open MIDI-Out port!');
}


This example opens a MIDI out port with the port number 2 (if this one is available). To get a list of available MIDI ports, just use MIDI.MidiOutList() or MIDI.MidiInList() respectively. As easy as this is, I quickly found out that you cannot open multiple ports and then, for instance, receive events from all the ports simultaneously. I tried different ways to circumvent this issue (e.g. calling MidiOutOpen() multiple times) but nothing really worked. That's quite a limitation.

On to the next step, receiving and sending MIDI events:

var inName = MIDI.MidiInOpen( 1, function(t, msg) {   
    if ((msg[0] != 254) && (msg[0] != undefined)) {
        MIDI.MidiOut(msg[0]+midiChannel-1, msg[1], msg[2]);   
    }
});


The above example opens the MIDI in port 1. Whenever a MIDI event comes along, it just triggeres the anonymous callback function which then again sends the MIDI event to the MIDI out port we just opened. The if-statement filters out all the MIDI "active sensing" events, which are generated by many devices just to tell the world that they are alive. Also, take a look at the MidiOut statement. It includes a global variable (bah!) called "midiChannel" which can hold the value 1 to 16 which indicate the "Midi Channel" to send the data to. A MIDI device, such as a synthesizer, will only receive the event, when it listens to the corresponding channel.

The next step, obviously would be to play a note. In MIDI each note is defined by two MIDI events called 'note-on' and 'note-off'. A MIDI note-of must be parametrized with a key value (which defines the pitch) and the key attack velocity (which most of the time defines the volume). Each can have valid values between 0 and 127. A note-off also has to have a key value and a release velocity (which is ignored by most MIDI devices). To play a note, you have to combine a note-on with a corresponding note-off event. The time difference between note-on and note-off defines the duration of the note. All this is accomplished by the following piece of code:

function playNote( note, velocity, length, channel ) {
        MIDI.MidiOut(144+channel-1, note, velocity);
        setTimeout( function() {
                MIDI.MidiOut(144+channel-1, note, 0);
                MIDI.MidiOut(128+channel-1, note, 64); // also send MIDI note off
            },
        length );
}


This sends a note-on on a defined MIDI channel (that's why it's 144+channel-1). And then it creates a timeout which will send the corresponding note-off 'length' milliseconds after the note-on.

Great, now we can play music on our devices triggered by JavaScript. But what happens if something goes awry and notes keep playing on the MIDI device forever? For this scenario MIDI inventend the 'all-notes-off' command, which tells the device to just shut up. Of course, we can send that from JavaScript as well, as the following example shows:

function sendAllNotesOff() {
    //loop over all MIDI channels and send all notes off and all controllers off
    for (var i=0; i<16; i++) {
        MIDI.MidiOut( 176+i, 123, 0 );   
        MIDI.MidiOut( 176+i, 121, 0 );   
    }
}


The following screenshot shows the MIDI events captured by the node.js program when playing on the MIDI keyboard.

MIDI Events
Have fun.

Wednesday, April 8, 2015

Implementing a REST API with node.js

Every so often there is the necessity to implement some mock-up web services or maybe even a prototype which shall offer its functionality via web services. This happened to me once again in March 2015 when preparing the API for a hackathon. To be precise, there already was an API, but it did not really fit to the requirements of the hackathon’s sponsor. Therefore, we decided to implement a second API (as a mock API) on top of the existing one. Our, second, API should add additional functionality to the existing services.
It was quite clear that the API should be REST with JSON as transport protocol. Unfortunately, we only had less than 4 weeks to design, implement and test the mock API. Thus, only a very lightweight technology could make this possible, like node.js.
Being JavaScript on the server, node.js is indeed a very lightweight technology. In its most basic configuration it will require you to write only one JavaScript file, which is then read and executed by node. Furthermore, it come with a module concept which lets you include already existing modules into your code with the “require” statement. Node also brings a packet manager (named “npm”) which lets you easily install all those modules to your project’s file system. In our mock API we used the following imports:
var http = require('http');
var express = require('express');
var url = require( "url" );
var path = require("path");
var fs = require("fs");
var queryString = require( "querystring" );

For our purposes, the “express” module is very important: it enables us to write REST services like in the following example. This short piece of code implements a web service, using the HTTP “GET” method, which echoes all input back to the response.

var api = express();

api.get('/echo', function(req,res) {

       var myobj = "";
      
        // parses the request url
        var theUrl = url.parse( req.url );
       
        //check if there is a query       
        if ((theUrl.query != null) && (theUrl.query != "")) {

             // gets the query part of the URL and parses it creating an object
             var queryObj = queryString.parse( theUrl.query );
              
             // queryObj contains the data of the query as an object
             // and jsonData is a property of it
             myobj = JSON.parse( queryObj.jsonData );                   
       }

       res.json( myobj );
});

As this examples shows, it’s really quite simple. For the response, you have to create a JavaScript object (which also might be a quite big one with arrays included and so on…) and transform it to JSON with “res.json”. This data is then returned to the client. There is a catch, though: with a HTTP “GET” method the input parameters will be encoded in the URL and therefore there is an upper limit of data which can be input to the service (around 8 kB, depending on the browser and other factors).  So, in order to potentially input more data into the web service, a “POST” method should be used, like in the next example. This one also shows how to extract parameters from the request header and do some basic authentication. Anyway, please bear in mind, it's still only a prototype source code (e.g. there should be another return code than 400 if an error occurred).

api.post('/api/booking', function (request, response) {

     //Read header & body
     var jsonData = request.body.requestPayload;

     var requestID = request.header('requestId');
     var userName = request.header('userName');
     var userPassword = request.header('password')
     var timeStamp = request.header('timeStamp');

     if (!checkAuthWithResponse(userName, userPassword, response)) return;

     resultAsString = '';

     var err = oVali.runHeaderValidation(request);
     if (err.length > 0) {
  
          // some logging and error handling here
     } else {

          // some business logic here

     }

     Response.send( resultAsString, 400);
})

Our mock API was intended to implement additional business logic on top of the existing API. The client would then call the mock API’s services to execute this business logic. Because the logic would only be depending on the values of the input parameters of the services there was no need for additional data storage or caching. Nevertheless, it would be quite easy as well to integrate a NoSQL database into node.js, for example.
Our mock API based on node.js was implemented just in time for the Hackathon and it was used there with very good results. We experienced no outages and had no problems with the performance as well.

As a summary, what are the advantages of using node.js for REST API development? We can name the following:

  1. As a developer, you are up and running very quickly. Install node, download some modules, write ONE file, and you are done.
  2. You don’t need to learn a lot of tools and IDEs (low overhead).
  3. The feedback cycles are very short, enabling you to develop with high speed.
  4. Integration with file system and databases is easy.
  5. Many standard problems can be used out-of-the-box with only a few lines of code.
  6. There is a big community, helping you out if you run into problems. Online documentation and FAQ is huge.
  7. It’s very easy to transport node projects. Just zip and unzip the files and you are done.
  8. There are a lot of possibilities to deploy and run node projects on the web. Pricing starts very low as well.

The bottom line is: node.js is a perfect choice for implementing prototype REST services. In how far this could be extended to full blown operational services – that’s another matter.

Thursday, March 19, 2015

Top Ten Tips for Hackathon Organizers

Want to organize a Hackathon? Great idea! A Hackathon is an amazing opportunity to collect new ideas, learn new stuff and get to know interesting people. Here, I'd like to share with you my personal experince from past Hackathons.

1. Have a goal
Communicate the objectives and goals you want the participants to achieve clearly and without room of interpretation. Tell them your criteria to judge if the the goal has been reached. That will enable the participants to think in the right direction and it will help you to easily find the winning team. Whether its to try out your API or to construct some fascinating algorithm - write it down and stick to it. BTW, also tell them about the prices as well :-)

2. Know the technology
You really have to be knowledgeable about your technology. And: you must be able to explain it to the participants. Tell them which technology to use, how to use it and what bugs and issues are known. They don't have to try it all out for themselves - that will only lead to frustration - and time is short. You really have to know your ropes concerning APIs, protocols, data models and frameworks. Be prepared to answer questions, so better think before, which might come up.

3. Built a protoype
There is nothing better to prepare for a Hackathon as to do everything you expect from the participants yourself in advance. Of course, you might not be as fast as they're gonna be, and you might not have a UI so pretty - but hack some code and try it all out, because only then will you really understand the relevant challenges with the technology. An then, if your prototype is ready, bring it and use it for reference.

4. Keep it simple
Time is running twice as fast as normal at a Hackathon. Or maybe even faster. So, nobody will find it amusing to read through handbooks to understand you challenge. So, keep it straight and simple. Remember, people are there to have fun and hack away freely. If possible only make them use ONE API not many, and only ONE protocol or data format. Use JSON and REST, instead of XML and SOAP. If possible leave out authentication. And avoid any extra overhead like having them deploy every hour or having them use a unknown tool.

5. Write documentation with sample code
You should write documentation for everything you want the participants to use, that will keep them from asking you everytime, The documentation should be proof-read to be as error-free as possible. It really has to be, so stay focused to keep if correct. And insert code snippets into the documentation which can be copy-pasted into their code and which should run out-of-the-box. Beware, some programs have problems with copying to the clipboard, like some acrobat versions, which distort the characters.

6. Bring all the stuff
You should not turn up there empty-handied, but rather carry a real large bag. You should bring a lot of marketing material, like banners, stand-up figures, mugs, T-shirts, stickers and everything a developer might love. Also bring all technology you might depend on, like projectors, adapters, notebooks and presentation software. If you need your laptop with the old VGA output, bringt the corresponding adapter.

7. Plan time before and after the event
Your technology and all relevant documentation should be up and running latest one week before the show. The participants just need some time to prepare themselves. And then, after the event let it all stay up and running for at least a week as well, because thedevelopers like to show off and tell their friends and collegues about it.

8. Dont show up there in a suit
And a tie is no good either. But bring a sleeping bag.

9. Collect the results
Be prepared to gather everything they produced at the event. You will need some time for that and better before the final presentation. So there should be dedicated people going around to collect all stuff. Perhaps you need a repository for that. And if you cannot get the code or the running system, then let them provide at least screenshots and/or movies - and their contact data.

10. Know your data
I experienced, that most teams show up at the Hackathon with a firm idea of what to build already in mind. And for this idea, they might need a certain dataset. So, if you cannot tell them how your data is structured and what the limits are, they won't have a good time. Be prepared to alter the dat if necessary or even to import extra data at the event.

Hackathon - Definition

Tuesday, September 23, 2014

Devoxx vs. JavaZone

Every now and then I get queried which of the Java and Web conferences in Europe I would recommend. Often, the most sensible answer seems to be: “the one closest to you”.
Nevertheless, since I visited JavaZone in Oslo in September 2014, I think it may be time to make a thorough comparison after all.
But which are the criteria? The number of rock star speakers? The volume of soft drinks available for free? Alas, I came up with the following criteria:

Variety of topics
This is especially important because most people visit conference in order to peak over their horizons or to deep-dive into certain areas. As a minimum, conferences should cover the topics architecture, mobile, methods, coding and research, while all might bear different names or might be slit up into several sub-categories.
Quality of talks
Of course, the presentations must have content and meaning and they must be given in an inspirational manner as well. These are high expectations, and not many can live up to them. Especially talks, which center around products or have a high marketing potential, tend to be boring and to be of low quality.
Choice & Availability
If you have picked up your favourite talk and you can’t see it, because the room is crammed, that’s bad. It’s even worse if there is no sensible alternative, because not enough parallel tracks are available.
Community feeling
You also visit a conference in order to meet other developers and to talk to fellow architects. This is enhanced if the conference introduces a feeling of community into all its visitors. This can be achieved via events which include all or many visitors, by special after-dinner events, by talks which address all of the community and so forth.
Organisation & Infrastructure
You need to find your way to the conference, you have to know the program, must find the correct room and you need a working wi-fi. Especially the last issue is always a source of trouble, but it’s getting more and more important every year.
Venue & after-conference program
If you want to enjoy your conference, you need good lighting, cosy seating, and a venue which makes you want more of it. And in the night you need some distraction from all the hard-brained conference stuff, in order to be fit again next day.
Food & beverages
Everyone who ever organized a party knows: food is the one single most important thing. Thre is a saying “full belly does not easily study”, but that’s even more true for an empty stomach.

For a comparison, I gave each of the conference between 0 and 5 points for each of the criteria. The result looks like this this:

Criterion
Devoxx
JavaZone
Variety of topics
4
3
Quality of talks
4
4
Choice & Availability
4
2
Community feeling
5
4
Organisation & Infrastructure
5
5
Venue & after-conference program
4
4
Food & beverages
2
5

Let me explain, how I came up with the numbers:
Variety of topics: Devoxx is in lead, because it covers nearly any topic one can think of. JavaZone is only shortly behind since the program also is very versatile.
Quality of talks: The quality is above average for both conferences. Due to the sheer amount of sessions, there are in absolute numbers more mediocre talks at Devoxx, but this does not lead to a full point more for JavaZone.
Choice & Availability: Devoxx is clearly better here because all of its talks are in English (JavaZone has Norwegian talks still). Furthermore, the conference is longer (3 vs. 2 days) and contains more parallel sessions. So, you’re in for some hard decisions at Devoxx. Nevertheless, Devoxx does not earn the full score, because it is so over crowded, that you cannot see all the talks you want.
Community feeling: JavaZone is a true community conference. The same is true for Devoxx, which is a little bit ahead, because there are Keynotes (not at JavaZone) and opening and closing talks which appeal to the whole of the visitors.
Organisation & Infrastructure: This is excellent at both conferences and also includes the awesome websites and the (mostly) working wi-fi.
Venue & after-conference program: Devoxx can gather further points here because it offers an evening program at two days. But this is reduced by the venue being too much outside of the city center.
Food & beverages: Food is traditionally poor at Devoxx and excellent at JavaZone.  Devoxx has 2 points, because of its Belgium Fries night.

Which one of the conferences to prefer? It depends on which one is closest…

Tuesday, November 19, 2013

Devoxx 2013 - The coming of lambda

It’s November 2013 and it’s conference time again. The European Java and Web Community gathered in Antwerp for the annual DEVOXX conference, which always means days tightly packed with information. My head is still spinning but I’d like to highlight a few things which are noteworthy this year. But first, a picture from the first keynote session, showing live-DJ-ing with software scripting:


No single hype

This year, there was no unique theme standing out. The hypes of the years past, Mobile, Cloud, NoSQL, BigData, are now regarded as standard. There were still presentations about those topics, but they are “normal” now. This of course is a good thing, although great emotions and extraordinary news are missing.


Java 8 expected with Lambdas and support for parallel execution

There were a lot of talks on Java 8 and on the new features available with Lambdas and functional / declarative programming.  They especially seem to enhance code readability which is important because “reading code is more important than writing code” (Brian Goetz). The most prominent example is the replacement of the infamous anonymous inner classes through lamdbas. This can be seen in the following code sample:

public class MyListener {
  public static void main(String[] args) {
    JButton myAnonymousButton = new JButton("A Button");
    
    //actionlistener using anonymous class
    myAnonymousButton.addActionListener(new ActionListener() {
      public void actionPerformed(ActionEvent ae) {
        System.out.println("anon click!");
      }
    });
    
    //actionlistener using lambda expression   
    myAnonymousButton.addActionListener(e -> { System.out.println("a lambda click!");
    });

    JFrame frame = new JFrame("Functional Sample");
    frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    frame.add(myAnonymousButton, BorderLayout.CENTER);
    frame.pack();
    frame.setVisible(true);
  }
}

Internally, the lambdas are implemented using invokedynamic which was introduced in Java 7. By doing so, the JVM can choose that implementation of the lambda which it sees fit. This is an advantage, since at programming time you don’t necessarily know the concrete circumstances of the runtime environment. On the other hand, the programmer loses control, because he only tells the JVM what to do but not how.
There is also a new stream library coming, which especially intends to simplify parallel programming. I’m not sure how much of a performance gain this can achieve, because it half of the code can be parallelized, the program can only run twice as fast.


Microsoft is Sponsor now           

The Devoxx organizers can be happy, because the “big four” are sponsors now: Google, Oracle, RedHat and Microsoft. Google dominated the talks, while, strangely enough, Microsoft did not show up in any of the presentations (although they had a booth). This is really a chance missed, especially since Windows 8 opened up to the Web and allowed for Windows Applications being written in JavaScript and HTML5.


More crowded, but…

Devoxx 2013 was even more crowded than the conferences of the last years. It was harder to get a seat for the important presentations and it was a pain to get something to eat. This all is ok, if the costs are low (which is true) and if the quality of the presentation content is high (which is not so true anymore). Actually, the quality of the slides was a catastrophy! I cannot remember another conference in the last years where “death by bullet point” was so imminent. Many presenters even read all their slides aloud and some did seem to be on the edge of sleeping (or at least trying to get their listeners to sleep). What’s more, good talks on methodology and architecture were missing, while many presenters focused on detailed features of various (sometimes not so important) frameworks. I also missed great keynote speakers. To wrap it up: quality at Devoxx seems to be on the decline. Maybe it’s because there are a lot of Devoxxes out there now (London, Paris, Kids …). Do they lose focus?


Highlights

Nevertheless, there were some very interesting and entertaining presentations: The session on Google Glass was absolutely packed and really lived up to the expectations. Another good one was Arun Gupta talking about web sockets. Also, Brian Goetz on details of the JVM and Ludovic Champenois on Google App Engine excelled. Next time, get us more of those, please.