Unofficial Erlang Planet

February 09, 2010

Damien Katz

First week in the new office

Last week was our first week in our new office in Old Downtown Oakland. It's a really neat area with lots of restaurants and bars, and hardly any murders.

Oh yeah, we've changed our name to Couchio. Our new blog will be here http://blog.couch.io/ soon.

Our office:
office - 4.jpg

Our office manager Claire:
office - 6.jpg

Chris and Mikeal:
office - 1.jpg

Claire and Jan:
office - 2.jpg

Nitin:
office - 8.jpg

Super Awesome Art by Julie Armbruster:
office - 3.jpg

Me:
office - 7.jpg

My Office:
office - 5.jpg

So far we are really disorganized and discombobulated. But I'm are working on it! I even bought Management for Dummies. Things will be running smoothly in no time ;)

Also we are looking hard for someone to help us offer CouchDB support and hopefully build a whole support organization. Email me damien@couch.io if you are interested or know someone who is.

by Damien Katz at February 09, 2010 12:40 AM

February 08, 2010

Debasish Ghosh

Scala Self-Type Annotations for Constrained Orthogonality

I talked about orthogonality in design in one of my earlier posts. We had a class Address in Scala and we saw how we can combine it with other orthogonal concerns without polluting the core abstraction. We could do this because Scala offers a host of capabilities to compose smaller abstractions and build larger wholes out of them. A language is orthogonal when it allows such capabilities of composition without overlaps in functionalities between the composing featuresets.

The design of Scala offers many orthogonal features - I showed some of them in my earlier post. The power of mixins for adding orthogonal features ..

val a = new Address(..) with LabelMaker {
  override def toLabel = {
    //..
  }
}

and the power of Scala views with implicits ..

object Address {
  implicit def AddressToLabelMaker(addr: Address) = new LabelMaker {
    def toLabel =
    //..
  }
}

Here Address and LabelMaker are completely unrelated and offers truly orthogonal capabilities when mixed in. However there can be some cases where the mixins themselves are not completely orthogonal to the core abstractions, but really optional extensions to them. In fact the mixins implement some functionalities that may depend on the core abstraction as well. Let's see yet another feature of the Scala type system that makes this modeling wholesome.

Consider the following abstraction for a security trade that takes place in a stock exchange ..

// details ellided for clarity
case class Trade(refNo: String, account: String, 
                 instrument: String, quantity: Int,
                 unitPrice: Int) {
  // principal value of the trade
  def principal = quantity * unitPrice
}


For every trade executed on the exchange we need to have a set of tax and fees associated with it. The exact set of tax and fees depend on a number of factors like type of trade, instruments traded, the exchange where it takes place etc. Let's have a couple of tax/fee traits that model this behavior ..

trait Tax { 
  def calculateTax = //..
}
  
trait Commission { 
  def calculateCommission = //..
}


In the above definitions both methods calculateTax and calculateCommission depends on the trade being executed. One option is to keep them abstract in the above trait and provide their implementations after mixing in with Trade ..

val t = new Trade(..) with Tax with Commission {
  // implementations
  def calculateTax = principal * 0.2
  def calculateCommission = principal * 0.15
}

I did it at the instance level. You can very well use this idiom at the class level and define ..

class RichTrade extends Trade with Tax with Commission {
  //..
}


However the above composition does not clearly bring out the fact that the domain rules mandate that the abstractions Tax and Commission should be constrained to be used with the Trade abstraction only.

Scala offers one way of making this knowledge explicit at the type level .. using self-type annotations ..

trait Tax { this: Trade =>
  // refers to principal of trade
  def calculateTax = principal * 0.2
}
  
trait Commission { this: Trade =>
  // refers to principal of trade
  def calculateCommission = principal * 0.15
}


The traits are still decoupled. But using Scala's self type annotations you make it explicit that Tax and Commission are meant to be used *only* by mixing them with Trade.

val t = new Trade(..) with Tax with Commission
t.calculateTax
t.calculateCommission


Can I call this constraining the orthogonality of abstractions ? Tax and Commission provide orthogonal attributes to Trade optionally and publish their constraints explicitly in their definitions. It's not much of a difference from the earlier implementations. But I prefer to use this style to make abstractions closer to what the domain speaks.

by Debasish (ghosh.debasish@gmail.com) at February 08, 2010 11:58 AM

February 02, 2010

Erlang Training and Consulting

02 February 2010: Erlang Solutions at QCon London 2010

Erlang Solutions Ltd. sponsors  QCon London 2010, hosts the track, gives the talks and organises Erlang User Group meeting there.

Qcon  is an annual London enterprise software development conference designed for team leads, architects and project management. It gathers Java, .NET, Ruby, SOA, Agile, Erlang and architecture communities.

Erlang Solutions will be present at QCon on Friday, 12th March 2010, when Ulf Wiger (CTO of Erlang Solutions) will be hosting the Concurrency Challenge track. He will also give an introductory talk The Concurrency Challenge at 10:20 and a presentation on Death by accidental complexity at 4:30 pm.

A day before, on 11th March 2010 from 18:30 until 20:30 Erlang Solutions organises special free London Erlang User Group Meeting at QCon. Everyone is welcome, even if you do not participate in QCon London 2010, just register here.There will be four talks:
  • Francesco Cesarini presents "Erlang community around the world"
  • Ulf Wiger talks about "Erlang in the Clouds"
  • Justin Sheehy will present "Introduction to RIAK" &
  • Joe Armstrong talks about "Erlang Libraries"
QCON 2010 will be held in The Queen Elizabeth II Conference Centre, in London from 8th until 12th March 2010.
 See you there!


Tutorials: March 8-9, 2010
Conference:  March 10-12, 2010

When registering for the QCon, use the Discount Code "erlangug" and save  £50 off the price!

February 02, 2010 03:16 PM

February 01, 2010

LShift

RabbitMQ-shovel: Message Relocation Equipment

In several applications, it’s very useful to be able to take messages out of one RabbitMQ broker, and insert them into another. Many people on our mailing list have being asking for such a shovel, and we’ve recently been able to devote some time to writing one. This takes the form of a plugin for Rabbit, and whilst it hasn’t been through QA just yet, we’re announcing it so people who would like to play and even suggest further features for inclusion can do so sooner rather than later.

The shovel is written on top of the Erlang client. It supports both direct and network connections to nodes, SSL support, the ability to declare resources on nodes it connects to, basic round-robinrabbit balancing of both source and destination nodes, and allows you to configure many parameters controlling how messages are consumed from the source, and how they’re published to the destination. Multiple shovels can be specified, their statuses queried, and shovels can repeatedly reconnect to nodes in the event of failure.

The plugin is available from http://hg.rabbitmq.com/rabbitmq-shovel/, and is released under the MPL v1.1. There is a README included which contains full documentation. This is replicated below.

RabbitMQ-shovel

Introduction

This is a plug-in for RabbitMQ that shovels messages from a queue on one broker to an exchange on another broker. The two brokers may be the same. The plug-in allows several shovels to be specified at the same time. Each shovel may have a number of source and destination brokers specified, and one of each is chosen whenever the shovel attempts to make a connection: this permits simple round-rabbit load balancing.

Resources can be declared upon connection to both the source and destination brokers, and parameters can be specified for both the reception and publishing of messages.

Requirements

Currently, you must build the server from source, under branch bug16653. You must also have checked out the rabbitmq-public-umbrella hg repository, and have the rabbitmq-erlang-client built. From scratch, the following commands should build RabbitMQ with the shovel plug-in:

hg clone http://hg.rabbitmq.com/rabbitmq-public-umbrella
cd rabbitmq-public-umbrella
hg clone http://hg.rabbitmq.com/rabbitmq-codegen
hg clone http://hg.rabbitmq.com/rabbitmq-erlang-client
hg clone http://hg.rabbitmq.com/rabbitmq-server
hg clone http://hg.rabbitmq.com/rabbitmq-shovel
cd rabbitmq-server
hg up -C bug16653
make -j
mkdir -p plugins
cd plugins
ln -s ../../rabbitmq-erlang-client
ln -s ../../rabbitmq-shovel
cd ../../rabbitmq-erlang-client
make
cd ../rabbitmq-shovel
make
cd ../rabbitmq-server
./scripts/rabbitmq-activate-plugins
make cleandb run

Configuration

The RabbitMQ configuration file specifies the shovel configurations. This exists by default, in /etc/rabbitmq/rabbitmq.config under Linux systems, %RABBITMQ_BASE%\rabbitmq.config under Windows or somewhere else under OS X. This file configures both RabbitMQ-server and all the plugins installed in it. It is an Erlang-syntax file of the form:

[{section1, [section1-config]},
 {section2, [section2-config]},
 …
 {sectionN, [sectionN-config]}
].

thus a list of tuples, where the left element of each tuple names the applications being configured. Don’t forget the last element of the list doesn’t have a trailing comma, and don’t forget the full-stop is needed after closing the list. Hence if you configure RabbitMQ-server and the RabbitMQ-shovel, then the configuration file may have a structure like this:

[{rabbit,        [configuration-for-RabbitMQ-server]},
 {rabbit-shovel, [configuration-for-RabbitMQ-shovel]}
].

A full example of the shovel configuration is:

{rabbit_shovel,
  [{shovels,
    [{my_first_shovel,
      [{sources,      [{brokers,
                          ["amqp://fred:secret@host1.domain/my_vhost",
                           "amqp://john:secret@host2.domain/my_vhost"
                          ]},
                       {declarations,
                          ['queue.declare',
                           {'queue.bind',
                                  [{exchange, "my_exchange">>},
                                   {queue,    
>}]}
                          ]}]},
       {destinations, [{broker, "amqp://"},
                       {declarations,
                          [{'exchange.declare',
                                  [{exchange, "my_exchange">>},
                                   {type, "direct">>},
                                   durable]}
                          ]}]},
       {queue, 
>},
       {qos, 10},
       {auto_ack, false},
       {tx_size, 0},
       {delivery_mode, keep},
       {publish_fields, [{exchange, "my_exchange">>},
                         {routing_key, "from_shovel">>}]},
       {reconnect, 5}
      ]}
     ]
   }]
}

Firstly, all shovels are named. Here we have one shovel, called ‘my_first_shovel’. We can have multiple shovels if you wish. Every shovel must have all sub-fields specified: sources, destinations, qos, auto_ack, delivery_mode, publish_fields, reconnect.

Sources and Destinations

Sources and destinations specify respectively where messages are fetched from and delivered too. One of ‘broker’ and ‘brokers’ must be specified, and ‘broker’ is simply shorthand for when only one broker needs specifying. Using ‘brokers’ allows a list of brokers to be specified: whenever the connection to a broker is lost, another one is chosen at random from the list and a connection attempt is made to that. The syntax for broker URIs is:

amqp://username:password@host:port/vhost

If username or password are omitted, the default values of guest and guest are used. If the vhost is omitted, the default value of / is used. If the host is omitted, then the plugin uses the “direct” connection internally rather than a network connection: this means it connects to the RabbitMQ-server node on which it is running without going through the network stack. This is much more efficient. If port is omitted then the default value is used (5672 or 5671 if SSL is used).

SSL is implemented, for which additional parameters are needed:

amqps://username:password@host:port/vhost?cacertfile=/path/to/cacert.pem&certfile=/path/to/certfile.pem&keyfile=/path/to/keyfile.pem&verify=verifyOption&fail_if_no_peer_cert=failOption

(note, this is a single line)

All five parameters (3 paths: cacertfile, certfile and keyfile; 2 options: verify, fail_if_no_peer_cert) must be specified. See the SSL guide at http://www.rabbitmq.com/ssl.html#configure-erlang for details of SSL in RabbitMQ in general and specifically for the Erlang client (on which the shovel is built).

Note that SSL cannot be used with the direct connection (i.e. a host must be specified when using SSL), and that it is preferable to use the non-SSL direct connection when connecting to the same node that’s running the shovel.

Resource Declarations

Both sources and destinations can have an optional ‘declarations’ clause. The value of this is a list, consisting of AMQP Methods. If default values are sufficient, then the method name alone can be specified - e.g. ‘queue.declare’. If parameters need to be set then the method should be given as a tuple, with the right hand side a proplist specifying which fields need altering from their default values. E.g:

    {'exchange.declare',[{exchange, "my_exchange">>},
                         {type, "direct">>},
                         durable]},

One very useful feature here is the Most-Recently-Declared-Queue feature, in which RabbitMQ remembers the name of the most recently declared queue. This means that you can declare a private queue, and then bind it to exchanges without ever needing to know its name.

queue :: binary

This parameter specifies the name of the queue on the source brokers to consume from. This queue must exist. Use the resource declarations to create the queue (or ensure it exists) first. Note again that the Most-Recently-Declared-Queue feature can be used here, thus an anonymous queue can be used: use <<>> to indicate the Most-Recently-Declared-Queue.

qos :: non-negative-integer

The shovel consumes from a queue. The QoS controls how many messages are sent to the shovel in advance of the message the shovel is currently processing.

auto_ack :: boolean

Setting this to ‘true’ turns on the no_ack flag when subscribing to the source queue.

tx_size :: non-negative-integer

When set to 0, transactions are not used. Other values make publishes transactional, with a commit every N messages. In lieu of the auto-ack option, when transactions are not used, messages are acknowledged to the source immediately after every publish. When transactions are used, acks are only issued to the source on receipt of the commit-ok message from the destination. This can thus be used to guarantee that messages are only acknowledged (and thus forgotten about by the source broker) when they are guaranteed to have been received by the destination broker.

delivery_mode :: ‘keep’ | 0 | 2

This affects the delivery_mode field when publishing to the destination. A value of ‘keep’ means that the same delivery_mode should be used as when the message was originally published to the source broker. 0 and 2 override the original setting.

publish_fields

This is a list of tuples which override fields in the publish method when publishing to the destination. This can be used to direct messages to a particular exchange on the destination, for example, or change the routing key. By default, the routing key of the message as it is received by the shovel is passed through, but this can be overridden as necessary.

reconnect :: non-negative-integer

When an error occurs, the shovel will disconnect from both the source and destination broker immediately. This will force uncommitted transactions at the destination to be rolled back, and delivered but unacknowledged messages from the source to be requeued. The shovel will then try connecting again. If this is unsuccessful, then it’s not a good idea for the shovel to very quickly and repeatedly try to reconnect. The value specified here is the number of seconds to wait between each connection attempt.

Note that if set to 0, the shovel will never try to reconnect: it’ll stop after the first error.

Obtaining shovel statuses

From the broker Erlang prompt, call rabbit_shovel_status:status(). This will return a list, with one row for each configured shovel. Each row has three fields: the shovel name, the shovel status, and the timestamp (a local calendar time of {{YYYY,MM,DD},{HH,MM,SS}}). There are 3 possible statuses:

  • ’starting’: The shovel is starting up, connecting and creating resources.

  • ‘running’: The shovel is up and running, shovelling messages.

  • {’terminated’, Reason}: Something’s gone wrong. The Reason should give a further indication of where the fault lies.

by matthew at February 01, 2010 12:13 PM

January 30, 2010

Steve Vinoski

Erlang Factory SF Bay Area 2010

Interested in Erlang? You might consider attending Erlang Factory SF Bay Area 2010. Below is a message that Francesco Cesarini, conference organizer and co-author of the most excellent book Erlang Programming, sent to the erlang-questions list yesterday providing more information about the conference, especially pointing out that the Very Early Bird registration deadline is tomorrow (Sunday January 31). Hope to see you there!

Hi All,

a note to say that we are almost done with the programme for the 2010 SF Bay Area Erlang Factory. This year, we are lucky to have keynote speakers such as Joe Armstrong, Bjarne Dacker, Kenneth Lundin and Steve Vinoski. They will be giving four of the 35 scheduled talks on the 25th and 26th of March in the San Francisco Bay Area. The almost complete programme is available here:

http://erlang-factory.com/conference/SFBay2010/programme

The conference will be preceded by three days of University courses taught by experts such as Simon Thompson, John Hughes, Thomas Arts, Henry Nystrom and Kevin Smith. Come and learn Erlang, OTP, QuickCheck or Web Development with Erlang. More information on the courses are here:

http://erlang-factory.com/conference/SFBay2010/university

This *Sunday* (January 31st) is the deadline for the very early bird deadline. Register by Sunday night and save $400 on the on-site registration price.

The conference hotel and venue is the SF Airport Hilton, a short BART / Caltrain ride from SF and the Valley. We have secured a very competitive price of US$109 per room and night at the conference hotel, this being one of the reasons for us choosing it. The other is the lower price of the venue, allowing us to pass on the savings to the delegates through a higher very early bird discount. We are planning an ErlLounge open to everyone who can’t make the two days on the 25th, and hope we will be able to surpass last year’s success. If you have thoughts or questions, you are welcome to drop me a line.

Hope to see you all there!

Francesco

http://www.erlang-solutions.com

by steve at January 30, 2010 09:29 PM

Erlang Inside

Erlang Factory SF Bay 2010 – Francesco Cesarini on the Conference and Old-School vs New-School Erlangers

The Erlang Factory’s 2010 conference is March 25th and 26th in San Francisco, with the university three days before, starting the 22nd.  The conference is at the Hilton San Francisco Airport, with three tracks, each on a different theme. The number of tracks gives attendees an unusually broad set of choices for talks. When attending [...]

by Chad DePue at January 30, 2010 08:03 PM

January 22, 2010

LShift

Plugin exchange types for RabbitMQ

An obvious extension point for an AMQP broker is the addition of new types of exchange. An exchange type essentially represents an algorithm for dispatching messages to queues, usually based on the message’s routing key, given how the queues are bound to the exchange — it’s a message routing algorithm.

At a minimum, supporting new exchange types requires only some scaffolding to plug in to (an exchange type registry) and a hook for routing messages. However, this wouldn’t support some more interesting use cases, and in particular it didn’t support our motivating use case. Exchange types that want to keep their own state need to be initialised, and be notified about other lifecycle events.

The branch bug22169 of RabbitMQ supports plugin exchange types, by providing a behaviour for exchange type modules to implement, and an exchange type registry to map a module to a type (i.e., what the client supplies in the type field of exchange.declare).

The behaviour requires exported hooks for validating exchange declarations, creating exchanges, recovering durable exchanges, publishing to an exchange (this is where the routing comes in), maintaining bindings, and deleting an exchange. RabbitMQ continues to maintain the database of exchanges and bindings, and calls the hooks after it’s done its own bookkeeping.

For simplicity, the hooks are not called atomically with the bookkeeping; so, it is possible for instance to publish to a new exchange for which the hook has not completed. However, provided there are no asynchronous operations in the hook implementation, the hook will have completed by the time the OK message is sent to the client. This is no more racey than AMQP itself, with consistency at the channel level where operations follow a single thread of control.

Here’s an example that simply io:formats things as they happen:

-module(rabbit_exchange_type_debug).
-include("rabbit.hrl").

-behaviour(rabbit_exchange_type).

-export([description/0, publish/2]).
-export([validate/1, create/1, recover/2, delete/2, add_binding/2, delete_binding/2]).
-export([register_debug_types/0]).
-include(”rabbit_exchange_type_spec.hrl”).

-rabbit_boot_step({debug_exchange_types, 
                   [{description, "debugging exchange types"},
                    {mfa, {?MODULE, register_debug_types, []}},
                    {requires, rabbit_exchange_type_registry},
                    {enables, exchange_recovery}]}).

description() ->
    [{name, <<"debug">>},
     {description, <<"Debugging exchange">>}].

backing_module(#exchange{ type = Type }) ->
    %% Presume that Type is EITHER one of the standard types –
    %% i.e,. that we have been registered this module as direct,
    %% topic, fanout or match — or, for testing purposes, it’s
    %% registered (as in the boot steps above) as debug_direct, etc.
    Type1 = case atom_to_list(Type) of
                “x-debug-” ++ T -> T; 
                              T -> T
            end,
    list_to_existing_atom(”rabbit_exchange_type_” ++ Type1).

publish(Exchange, Delivery) ->
    io:format(”Publish ~p to ~p~n”, [Delivery, Exchange]),
    Module = backing_module(Exchange),
    Module:publish(Exchange, Delivery).

validate(X) ->
    io:format(”Validate ~p~n”, [X]),
    (backing_module(X)):validate(X).

create(X) ->
    io:format(”Create ~p~n”, [X]),
    (backing_module(X)):create(X).

recover(X, Bs) ->
    io:format(”Recover ~p with bindings ~p~n”, [X, Bs]),
    (backing_module(X)):recover(X, Bs).

delete(X, Bs) ->
    io:format(”Delete ~p with bindings ~p~n”, [X, Bs]),
    (backing_module(X)):delete(X, Bs).

add_binding(X, B) ->
    io:format(”Add binding ~p to ~p~n”, [B, X]),
    (backing_module(X)):add_binding(X, B).

delete_binding(X, B) ->
    io:format(”Delete binding ~p from ~p~n”, [B, X]),
    (backing_module(X)):delete_binding(X, B).

register_debug_types() ->
    lists:foreach(
      fun (T) ->
              rabbit_exchange_type_registry:register(T, ?MODULE)
      end,
      [<<"x-debug-direct">>,
       <<"x-debug-topic">>,
       <<"x-debug-fanout">>,
       <<"x-debug-headers">>]).

Bit by important bit:

-behaviour(rabbit_exchange_type).
-export([description/0, publish/2]).
-export([validate/1, create/1, recover/2, delete/2, add_binding/2, delete_binding/2]).

rabbit_exchange_type specifies these exported callbacks.

-include("rabbit_exchange_type_spec.hrl").

This include has the specs for each of the exported functions, if you’re using specs.

-rabbit_boot_step({debug_exchange_types, 
                   [{description, "debugging exchange types"},
                    {mfa, {?MODULE, register_debug_types, []}},
                    {requires, rabbit_exchange_type_registry},
                    {enables, exchange_recovery}]}).

This uses the new boot sequence mechanism to register the exchange type during boot. The “enables” and “requires” say that the function given as mfa above must be run after the exchange type registry is available, but before any exchanges are recovered.

publish(Exchange, Delivery) ->
    io:format("Publish ~p to ~p~n", [Delivery, Exchange]),
    Module = backing_module(Exchange),
    Module:publish(Exchange, Delivery).

This exchange type simply delegates to a “backing” exchange type.

register_debug_types() ->
    lists:foreach(
      fun (T) ->
              rabbit_exchange_type:register(T, ?MODULE)
      end,
      [<<"x-debug-direct">>,
       <<"x-debug-topic">>,
       <<"x-debug-fanout">>,
       <<"x-debug-headers">>]).

rabbit_exchange_type_registry maintains a registry of type to module; because of this indirection, we can register this module as many different types, then check the declared type of the exchange in our hook to see which type we’re expected to be. Note that the AMQP specification requires the “x-” prefix for non-standard exchange types.

This should reach default branch soon after RabbitMQ 1.7.1 is released. Until then, if you want to play, you’ll have to

rabbitmq-server$ hg update -C bug22169

You can drop modules straight into src, but they are better packaged as plugins — follow the drill at the plugin development guide (and note that your plugin may only need to be a library application).

[EDITS: Updated sample code to keep up with name changes, and correct use of list_to_atom to list_to_existing_atom]

by mikeb at January 22, 2010 02:38 PM

January 21, 2010

Debasish Ghosh

A new way to think of Data Storage for your Enterprise Application

A couple of posts earlier I had blogged about a real life case study of one of our projects where we are using a SQL store (Oracle) and a NoSQL store (MongoDB) in combination over a message based backbone. MongoDB was used to cater to a very specific subset of the application functionality, where we felt it made a better fit than a traditional RDBMS. This hybrid architecture of data organization is turning out to be an increasingly attractive option today with more and more specialized persistent storage structures being developed.

In many applications we need to process graph data structures. Neo4J can be a viable option for this. You can have your mainstream data storage still in an RDBMS and use Neo4J only for the subset of functionalities for which you need to use graph data structures. If you need to sync back to your main storage, use messaging as the transport to talk back to your relational database.

Multiple data storage use along with asynchronous messaging is one of the options that will looks very potent today. Drizzle has its entire replication based on a RabbitMQ based transport. And using AMQP messaging, Drizzle replicates data to a host of key/value stores like Voldemort, memcachedDB and Cassandra.

If we agree that messaging is going to be one of the most dominant paradigms in shaping application architectures, why not try to go one level up and look at some higher level abstractions for message based programming? Erlang programmers have been using the actor model for many years now and have demonstrated all the good qualities that the model imbibes. Inspired by Erlang, Scala also offers a similar model on the JVM. In an earlier post I had discussed how we can use the actor model in Scala to scale out messaging applications using a RabbitMQ storage.

Now with the developing ecosystem of polyglot storage, we can use the same model of actor based communication as the backbone for integrating multiple data storage options that you may plug in to your application. Have specific clients front end the storage that they need to work with and use messaging to sync that up with the main data storage backend to have a consistent system of record. Have a look at the following diagram that may not look that unreal today. You have a host of options that bring your data closer to the way you process them in your domain model, be it document oriented, graph based, key/value based or simple POJO based across a data grid like Terracotta.



When we have a bunch of architectural components loosely connected through messaging infrastructure, you can have a world of options managing interactions between them. In fact your options open up more when you get to interact with data shaped the way you would like to be. You now can think in terms of having a data model aligned with the model of your domain. You know once your rule base gets updated in Neo4J, it will somehow be synced up with the backend storage through some other service that will make it eventually consistent.

In a future post I will explore some of the options that a higher order middleware service like Akka can add to your stack. With Akka providing abstractions like transactors, pluggable persistence and out of the box integration modules for AMQP, there's a number of ways you can think of modularizing your application's domain model and storage. You can use peer to peer distributed actor based communication model that sets up synchronization options with your databases or you can use AMQP based transport to do the same much like what Drizzle does for replication. But that's some food for thought for yet another future post.

by Debasish (ghosh.debasish@gmail.com) at January 21, 2010 07:24 PM

January 18, 2010

Mickaël Rémond

[ANN] ejabberd 2.1.2 bugfix release

We are pleased to announce ejabberd 2.1.2, which contains several bugfixes over last month's maintenance release.

Brief summary of changes:

  • Fix SASL PLAIN authentication message for RFC4616 compliance
  • Fix support for old Erlang/OTP R10 and R11
  • If server start fails due to config file problem, display some lines and stop node
  • PubSub and PEP: several improvements and bugfixes
  • WebAdmin: fix offline message displaying
  • When server stops with new stop_kindly command: inform users, clients, MUC

 

Check the Release Notes for a more complete list of changes:
http://www.process-one.net/en/ejabberd/release_notes/release_note_ejabberd_2.1.2

If you upgrade from ejabberd 2.0.5 or older, read carefully the release notes of ejabberd 2.1.0 too, because there were several changes in the installation path and the configuration options.

The list of solved tickets since previous version is available on ProcessOne bug tracker:
http://redir.process-one.net/ejabberd-2.1.2

ejabberd 2.1.2 is available as source code package and binary installers for Linux 32 bits, 64 bits, Mac OS X Intel, and Windows:
http://www.process-one.net/en/ejabberd/downloads

by Jérôme Sautret at January 18, 2010 04:08 PM

January 16, 2010

Curious Attempt Bunny

My First Arduino Hack

Yay! I got an Arduino as a present! So what do you do with it? I'm at DorkBotPDX and I'd just got as far as turning on an LED with a push button, and Ward suggested I try getting it to echo back to my a rhythm of clicks. I got that working, and then starting looking around in my starter kit. I had a speaker! Here's my circuit diagram and the code to go with it:

(The mystery component on the left is the buzzer - Fritzing didn't have a close match for that part)

The code:
#define BUZZER 3
#define BUTTON 7

int val = 0;
int old_val = 0;

long times[50];
long i = 0;

void setup() {
pinMode(BUZZER, OUTPUT);
pinMode(BUTTON, INPUT);
}

void loop() {
val = digitalRead(BUTTON);
if ((val == HIGH) && (old_val == LOW)) {
times[i] = millis();
i++;
analogWrite(BUZZER, 64);
delay(10);
}
if ((old_val == HIGH) && (val == LOW)) {
times[i] = millis();
i++;
analogWrite(BUZZER, 0);
delay(10);
}

old_val = val;

if (i > 0 && millis() - times[i-1] > 2000) {
delay(1000);
int state = 0;
for(int y=0; y<i; y++) {
state = 1 - state;
analogWrite(BUZZER, 64*state);
delay(times[y+1]-times[y]);
}
i = 0; // reset
analogWrite(BUZZER, 0);
}
}
Word to the wise: as with coding it pays to take small incremental steps!

by Curious Attempt Bunny (noreply@blogger.com) at January 16, 2010 04:58 PM

JLOUIS Ramblings

The state of Haskell-Torrent

Implementing processes

The Haskell Bittorrent project is evolving at a steady state these days. In the last couple of days, we have implemented most of the code relevant for carrying out choking of peers. Choking is the process by which you only communicate to a few peers at a time. Thus TCP/IP congestion can be avoided which drives up the download and upload rates. This post is not on this part however, which must wait a bit.

A fellow dane, Thomas Christensen, heeded my call and did some hlint runs over the code. Hopefully, we’ll see more work from him (github). Alex Mason has added even more parsing stuff through Cereal to the code, fixed a number of bugs in the parser and improved its general state. His venture is described on his blog (here).

Processes

I will be talking about processes in this post. When haskell-torrent started, our processes were simply “things spawned in the IO monad”. The approach works, but it quickly becomes inadequate for several reasons. In Haskell and FP in general, a lot of power stems from the idea that we can write a large set of small building blocks and then compose them together to form increasingly larger and larger blocks as we go. When composing, we use a fairly small number of helpers — readily present in the standard libraries.

When running in IO, we quickly end up with a lot of state. This can of course be passed around by “hand” and tail-calls. Unfortunately, this means we will end up using a lot of our precious coding time doing just exactly that. To optimize, we need a way to reflect away configuration and state when we don’t need it, and reify the information at certain points in the program where it is necessary to know the state.

The ubiquitious tool in Haskell for this are Monads. Not a single monad like IO, but a for-the-case relevant monad built from monad transformers. A couple of days ago, I installed XMonad by Spencer Janssen, Don Stewart, Jason Creighton and many more. This is a window manager — but surprisingly, it needs to solve some problems similar to the one in haskell-torrent. By inspiration from XMonad, we define

> newtype Process a b c = Process (ReaderT a (StateT b IO) c)
>    deriving (Functor, Monad, MonadIO, MonadState b, MonadReader a, Typeable)

That is, a Process is a type. It contains some configuration data a, an internal state b and is in the process of evaluating to a value of type c. We use a Reader transformer so we can ask for configuration data when we need it. We do not expect this configuration data to be altered when the process runs. A State Transformer takes care of the internal state of the process. Finally, we let the underlying monad be IO. This gives us access to CML and the outside world.

We let GHC derive a large set of things automatically (using several extensions in the process). This gives an easier way to manipulate the state when the process is running.

Running a Process is easy:

> runP :: a -> b -> Process a b c -> IO (c, b)
> runP c st (Process p) = runStateT (runReaderT p c) st

which is exactly like in XMonad. Spawning a new process is also fairly easy:

> spawnP :: a -> b -> Process a b () -> IO ThreadId
> spawnP c st p = spawn proc
>   where proc = do runP c st p
>                   return ()

In general different processes will have different configurations a. These will usually contain the channels on which the process can communicate. We then define a type class

> class Logging a where
>   getLogger :: a -> LogChannel
> 
> instance Logging LogChannel where
>   getLogger = id

of types that contain a logger channel. This means we can define a generic log function like this:

> log :: Logging a => String -> Process a b ()
> log msg = do
>     logC <- asks getLogger
>     liftIO $ logMsg logC msg

Type classes are a magnificent tool when we want to coerce a general function on top of different types. Many of our configurations in the client will instance the Logging class and then the log function knows how to access the logger in the Reader.

What does this buy us

The advantage of doing this change on the code is twofold: First, the amount of parameter passing is considerably reduced. Reflection into the monad solves this problem. We are now able to compose easier. Function composition is considerably harder when there are many parameters abound. With the change, preliminary restructuring of the Peer process shows a much simpler flow. Also, there are now ample refactoring opportunities available with the change.

Second, the monad means we can use locality much more to our advantage. A common idiom is to modify the state of the process or to retrieve the current state for query. This now happens locally at the point where it is needed. Before, we might have needed to pass a parameter through several functions and then use it.

What next?

There are a small number of things that needs to be addressed before we can claim that the client is a full bittorrent client:

  • The client needs to correctly handle the concept of interest. It must tell other clients if it is interested in the pieces they have at their disposal for transmission. I have some preliminary code for doing this.
  • The client needs to correctly tell the tracker how many bytes it uploaded and downloaded. This measure is needed on many private trackers as they require people to upload data back.
  • The client needs to be better at choosing the next eligible piece. Choosing one randomly is good enough.
  • The client needs to handle multi-file torrents. It is not as hard as it may sound — the only part of the system that needs to know about files is the code handling the file system. All other parts can just keep on transferring pieces.
  • For choking to work correctly, we must know how fast we are currently transferring to a peer. This is an interesting little problem if somebody feels their curiosity tickled :)
  • We currently take space proportional to torrent size due to our SHA1 calculation being slow and not use a file descriptor. Research into a faster SHA1 library would be really beneficial.
  • We need to accept incoming connections. The system only connects outward at the moment.

And of course, it needs some testing in a non-lab setting. Currently it can seed and leech, but my setup is very simple: Opentracker and rtorrent on another computer.

The main github repository for haskell-torrent is here.

by Jlouis (noreply@blogger.com) at January 16, 2010 10:31 AM

January 14, 2010

Dukes of Erl

Minor Erlang Interface Tricks

erlrc requires that you drop a file whose name is your node name and whose contents is the node cookie into a particular directory so that the packaging system can find running Erlang VMs and ask them to do hot-code upgrades. It's not that hard, but I figured I would put some shell scripts into the erlrc google code project demonstrating how to do this and other minor tricks that make it a little nicer to talk to an Erlang VM from the (non-Erlang) command line.
All of these scripts are driven by a POSIX shell syntax configuration file describing an Erlang VM, which can be pretty short. Here's one I'm using right now for my personal stuff:
% cat /usr/share/myerlnode/myerlnode.rc
node_name_file='/etc/erlrc.d/nodes/erlang'

run_extra_args='+A 64 -noshell -noinput -s crypto -s mnesia -eval "case erlrc_boot:boot () of ok -> ok; _ -> init:stop () end" >erlang.out 2>erlang.err &'

node_name_file (the only required config setting) defines what the node name will be (via the basename). This config file drives four scripts:
  1. erlstart-run-erlang: starts an Erlang VM. doesn't do that much, really: creates the node file for erlrc, and sets the heart command in case you want to use heart.
  2. erlstart-remsh: starts a remote shell on an Erlang VM.
  3. erlstart-eval: takes the argument, evals it on an Erlang VM, and prints the result to standard out. very useful shell script glue for maintenance scripts.
  4. erlstart-etop: runs etop on an Erlang VM.
If you put your config file in the default location (/etc/erlstart.rc) or export an environment variable indicating the location (ERLSTART_CONFIG_FILE) then things are pretty zero-configuration. Doing this kind of thing at the (non-Erlang) shell gets very addictive:
% erlstart-eval 'application:which_applications()'
[{anwhereos,"Rest API for time series persistence and retrieval.","0.1.0"},
{drurlyjsclientsrv,"Serve the drurly jsclient from the drurly server.",
"0.0.1"},
{drurly,"Social sharing server.","2.2.0"},
{mcedemo,"TinyMCE + Nitrogen demo.","1.3.0"},
{inets,"INETS CXC 138 49","5.0.12"},
{mochiweb,"MochiWeb is an Erlang library for building lightweight HTTP servers.",
"0.2009.05.26"},
{nitrogen,"Nitrogen web framework for Erlang.","0.2009.05.12.3"},
{nitromce,"A Nitrogen element which corresponds to a TinyMCE editor instance.",
"4.0.1"},
{sgte,"String template language for Erlang.","0.7.1"},
{signzor,"Erlang library to generate signed printable encodings.","0.0.1"},
{tcerl,"Erlang driver for tokyocabinet.","1.3.1h"},
{webmachine,"An Erlang REST framework.","0.2009.09.24b"},
{erlrc,"Extensible application management.","0.2.3"},
{mnesia,"Mnesia storage API extensions.","4.4.7.6.1"},
{crypto,"CRYPTO version 1","1.5.3"},
{sasl,"SASL CXC 138 11","2.1.5.4"},
{stdlib,"ERTS CXC 138 10","1.15.5"},
{kernel,"ERTS CXC 138 10","2.12.5"}]
erlstart-run-erlang is fairly low level, so for an init script I would do something like the following: first, a little stub installed into /etc/rc.d
#! /bin/sh

# chkconfig: 2345 20 80
# description: Control my personal Erlang node.

exec myerlnodectl "$@"

and then the actual guts installed as myerlnodectl
#! /bin/sh

eval_with_main_node () \
{
erl -name myerlnodetmp$$ \
-hidden \
-setcookie "$cookie" \
-noshell -noinput \
-eval "MainNode = list_to_atom (\"$1\"), $2" \
-s erlang halt
}

get_hostname () \
{
erl -name myerlnodetmp$$ -setcookie $$ -noshell -noinput -eval '
[ Host ] = tl (string:tokens (atom_to_list (node ()), "@")),
io:format ("~s~n", [ Host ])
' -s init stop
}

id=`basename "$0"`

if test -d /root
then
HOME=${HOME-/root}
else if test -d /var/root
then
HOME=${HOME-/var/root}
fi
fi
export HOME

ERLSTART_CONFIG_FILE=${ERLSTART_CONFIG_FILE-/usr/share/myerlnode/myerlnode.rc}
export ERLSTART_CONFIG_FILE

. "$ERLSTART_CONFIG_FILE"

cookie=${cookie-turg}
user=${user-erlang}
hostname=${hostname-`get_hostname`}
node_name=`basename "$node_name_file"`
full_name="$node_name@$hostname"
shutdown_file=${shutdown_file-/var/run/myerlnode.shutting_down}

ERL_CRASH_DUMP=${ERL_CRASH_DUMP-/dev/null}
export ERL_CRASH_DUMP

case ${1-"status"} in
start)
test "`id -u`" -eq 0 || exec sudo $0 "$@"

printf "Starting Erlang... "

if test -f "$node_name_file" && \
test true = "`erlstart-eval 'true' 2>/dev/null`" 2>/dev/null
then
echo "already started."
exit 0
fi

pid=`eval_with_main_node "$full_name" '
io:format ("~p", [
case rpc:call (MainNode, os, getpid, []) of
{ badrpc, _ } -> undefined;
Pid -> list_to_integer (Pid)
end ])'`

test "$pid" -gt 0 2>/dev/null && {
test -f "$shutdown_file" && {
oldpid=`cat "$shutdown_file"`
test "$pid" -eq "$oldpid" && {
echo "shutdown in progress (pid = '$pid')." 1>&2
exit 1
}
}
}

rm -f "$shutdown_file"

# check for -s shell support
su -l -s /bin/sh $user -c true >/dev/null 2>/dev/null

if test $? = 0
then
dashs="-s /bin/sh"
else
dashs=""
fi

${niceness+ nice -n $niceness} \
su -l $dashs "$user" -c \
"env cookie=\"$cookie\" erlstart-run-erlang \"$ERLSTART_CONFIG_FILE\""

eval_with_main_node "$full_name" \
"Wait = fun (_, 0) ->
failed;
(Cont, Max) ->
case net_adm:ping (MainNode) of
pong ->
ok;
pang ->
timer:sleep (100),
Cont (Cont, Max - 1)
end
end,
DontTellMe = 100,
ok = Wait (Wait, DontTellMe)" || {
echo "" 1>&2
echo "$id: could not connect to node after 10 seconds" 1>&2
exit 1
}

eval_with_main_node "$full_name" \
"Wait = fun (_, 0) ->
failed;
(Cont, Max) ->
case rpc:call (MainNode, init, get_status, []) of
{ started, _ } ->
ok;
{ starting, _ } ->
timer:sleep (100),
Cont (Cont, Max - 1);
{ Status, _ } ->
{ failed, Status }
end
end,
DontTellMe = 100,
ok = Wait (Wait, DontTellMe)" || {
echo "" 1>&2
echo "$id: node did not boot after 10 seconds" 1>&2
exit 1
}

echo "done."
;;

stop)
test "`id -u`" -eq 0 || exec sudo $0 "$@"

printf "Stopping Erlang... "

if test ! -f "$node_name_file" || \
test true != "`erlstart-eval 'true' 2>/dev/null`" 2>/dev/null
then
echo "not running."
exit 0
fi

pid=`erlstart-eval 'os:getpid ()' 2>/dev/null`

test "$pid" -gt 0 2>/dev/null && {
printf '%s' $pid > "$shutdown_file"
chown $user:$user "$shutdown_file"
}

erlstart-eval 'init:stop ()' >/dev/null 2>/dev/null

eval_with_main_node "$full_name" \
"Wait = fun (_, 0) ->
failed;
(Cont, Max) ->
case net_adm:ping (MainNode) of
pong ->
timer:sleep (100),
Cont (Cont, Max - 1);
pang ->
ok
end
end,
DontTellMe = 100,
ok = Wait (Wait, DontTellMe)" || {
echo "" 1>&2
echo "$id: node still responsive after 100 seconds" 1>&2
exit 1
}

rm -f "$node_name_file"

echo "done."
;;

status)
if test -f "$node_name_file" && \
test true = "`erlstart-eval 'true' 2>/dev/null`" 2>/dev/null
then
echo "Erlang is running"
else
echo "Erlang is not running"
fi

;;

*)
echo "$id: unknown command $1" 1>&2
exit 1
esac

exit 0

Hopefully you found that inspirational.

by Paul Mineiro (noreply@blogger.com) at January 14, 2010 12:41 PM

Steve Vinoski

WS-REST 2010 CfP

I’m on the program committee for the WS-REST 2010 conference. Please consider submitting a paper, but note that the submission deadline is coming up quickly.

WS-REST 2010
http://ws-rest.org/
Paper Submission: February 8, 2010

Call for Papers

The First International Workshop on RESTful Design (WS-REST 2010) aims to provide a forum for discussion and dissemination of research on the emerging resource-oriented style of Web service design.

Background

Over the past few years, several discussions between advocates of the two major architectural styles for designing and implementing Web services (the RPC/ESB-oriented approach and the resource-oriented approach) have been mainly held outside of the research and academic community, within dedicated mailing lists, forums and practitioner communities. The RESTful approach to Web services has also received a significant amount of attention from industry as indicated by the numerous technical books being published on the topic.

This first edition of WS-REST, co-located with the WWW2010 conference, aims at providing an academic forum for discussing current emerging research topics centered around the application of REST, as well as advanced application scenarios for building large scale distributed systems.

In addition to presentations on novel applications of RESTful Web services technologies, the workshop program will also include discussions on the limits of the applicability of the REST architectural style, as well as recent advances in research that aim at tackling new problems that may require to extend the basic REST architectural style. The organizers are seeking novel and original, high quality paper submissions on research contributions focusing on the following topics:

  • Applications of the REST architectural style to novel domains
  • Design Patterns and Anti-Patterns for RESTful services
  • RESTful service composition
  • Inverted REST (REST for push events)
  • Integration of Pub/Sub with REST
  • Performance and QoS Evaluations of RESTful services
  • REST compliant transaction models
  • Mashups
  • Frameworks and toolkits for RESTful service implementations
  • Frameworks and toolkits for RESTful service consumption
  • Modeling RESTful services
  • Resource Design and Granularity
  • Evolution of RESTful services
  • Versioning and Extension of REST APIs
  • HTTP extensions and replacements
  • REST compliant protocols beyond HTTP
  • Multi-Protocol REST (REST architectures across protocols)

All workshop papers are peer-reviewed and accepted papers will be published as part of the ACM Digital Library. Two kinds of contributions are sought: short position papers (not to exceed 4 pages in ACM style format) describing particular challenges or experiences relevant to the scope of the workshop, and full research papers (not to exceed 8 pages in the ACM style format) describing novel solutions to relevant problems. Technology demonstrations are particularly welcome, and we encourage authors to focus on “lessons learned” rather than describing an implementation.

Papers must be submitted electronically in PDF format. Submit at the WS-REST 2010 EasyChair installation.

Important Dates

  • Submission deadline: February 8, 2010, 23.59 Hawaii time
  • Notification of acceptance: March 1, 2010
  • Camera-ready versions of accepted papers: March 14, 2010
  • WS-REST 2010 Workshop: April 26, 2010

Program Committee Chairs

Program Committee

Contact

WS-REST Web site: http://ws-rest.org/
WS-REST Email: chairs@ws-rest.org

by steve at January 14, 2010 05:45 AM

January 11, 2010

Erlang Training and Consulting

14 December 2009: Erlang Training and Consulting's 10th Birthday – it's time to change our name!

2009 has been a huge year for us. Not only have we had an incredible year assisting companies on six continents with their use of Erlang but we have also organised 4 conferences, 10 Erlang User Group Meetings and  started a KTP project in e-learning with the University of Kent. To add to all that - it’s also our 10th Birthday!

It all began 10 years ago on 13th December 1999, when our founder Francesco Cesarini decided to set up the first non-Scandinavian company to offer Erlang/OTP training and consultancy services. At that time, Erlang wasn't known outside of the 'Ericsson world'. Over the years the business grew dramatically. We have expanded geographically, with offices now in London (UK), Uppsala (Sweden) and Krakow (Poland) and now we employ 40 employees worldwide. The services on offer have also expanded significantly. We are now a one-stop shop for all Erlang needs, including:
A year ago we started running the Erlang Factory conferences. These are aimed at all levels of the development community including Architects, Programmers, Project Managers, Software and Platform Innovators as well as existing Erlang users. They are also designed to present Erlang to prospective users, Erlang newbies and anyone else interested in making use if the many strengths of the language. The Erlang Factory takes place twice a year - in the San Francisco Bay Area in March/April and in London in June. In 2009 we also took over the organisation of the Erlang User Conference in Stockholm – the oldest Erlang conference in the world, which has been held annually since 1994. Finally this year the first Erlang day in Krakow (Poland) called Erlang Factory Lite was held to grow interest in Erlang of the substantial technology Community in that country.

Great service, a customer-focus and skilled and experienced staff have been the basis of our growth over the past ten years. This will not change. However in 2010 we will do it with a new name! In order to better describe the breadth of our service offering, on 1st January 2010 we will become Erlang Solutions Ltd.

January 11, 2010 04:32 PM

09 December 2009: Marcus Taylor at Strengthen Your Business

More than 50 representatives from the business world attended a special networking event at the University of Kent’s Medway campus.  Called Strengthen Your Business, the free session on 25 November gave business leaders the chance to explore how their companies can team up with the University, and benefit from its commercial expertise and resources.

A variety of business people who had successfully worked with the University on student placements or Knowledge Transfer Partnerships spoke of their experiences, while others outlined the opportunities for collaboration in graduate recruitment and continuing professional development schemes. There was also the chance for individuals to network after the main talks.

Marcus Taylor, Chief Executive for London-based Erlang Training and Consulting, spoke of the strategic help his company had received with Knowledge Transfer Partnerships – a government-funded initiative that helps businesses develop their competitiveness, productivity and performance. ‘I’m not an experienced bid writer, but the University has a wealth of experience in this area and supported us wonderfully throughout the whole process – so much so that we’re thinking of applying for our fourth project,’ he said.

               

James Corbin, Placement Officer for the University of Kent, said it was a golden opportunity for businesses to find out about the resources on their own doorstep. 'The University is host to a wealth of knowledge that can often provide businesses with the solutions they seek,’ he said. ‘We’re convinced that firms who work with the University can gain a competitive edge.’

The event followed hot on the heels of the launch of the University’s ICE initiative, which took place at the Canterbury Innovation Centre on 17 November. ICE stands for innovation, creativity and enterprise, and is a project that aims to connect local, regional, national and international businesses to the University of Kent and to each other.

The Strengthen Your Business event at Medway was run in partnership with Thames Gateway Chamber of Commerce.

January 11, 2010 04:32 PM

07 December 2009: Erlang Factory 2010 Dates Announced!!

The dates of the Erlang Factory in the SF Bay Area and London have been announced.
Registration will be opening soon and we are currently accepting talk proposals. If you want to give a presentation at the Erlang Factory, you can find more information how to do that, here. Keep a look out and follow us on Twitter @erlangfactory

.

January 11, 2010 04:32 PM

30 November 2009: Erlang Training and Consulting soon Erlang Solutions will be present at ACCU 2010

Francesco Cesarini (founder of Erlang Training and Consulting, soon Erlang Solutions Ltd.) and Ulf Wiger (CTO of Erlang Training and Consulting, soon Erlang Solutions Ltd.) will be speaking  at ACCU 2010. Francesco's talk on Styling your Architecture in an Evolving Concurrent World and Ulf's on Message-Passing Concurrency in Erlang, have both been accepted by the conference committee.



ACCU 2010 will take place from 14th to 17th April 2010  at the Barcelo Oxford Hotel, in Oxford, UK. The programme of the conference will feature a special track on software testing, and sessions on concurrent and distributed systems, C++, Java,agile development and Erlang!

                                                                 Meet us at ACCU 2010!  

January 11, 2010 04:32 PM

23 November 2009: The Largest Erlang Open Source Project Directory!

Trapexit has announced its own Erlang open source crawler. It will crawl the web, gathering searchable information on all open source Erlang projects from all the major repositories. Trapexit will allow discussing these projects in the forum, adding documentation in the Trapexit wiki or rating the various contributions. To view the project index repository, visit http://projects.trapexit.org.

January 11, 2010 04:32 PM

19 November 2009: Learn Erlang from experts

Do you want to learn about  the basic, sequential and concurrent aspects of the Erlang programming language? And would you like to be taught by the expert, author of the 'Erlang Programming' book?

If the answer is yes, then we have something which may interest you:

On Wednesday, 3rd February 2010 in London  Francesco Cesarini, the author of 'Erlang Programming'  also the founder and CSO of Erlang Training and Consulting Ltd. will give a tutorial on Practical Erlang Programming.

You will learn the basics of how to read, write and structure Erlang programmes. The goal of tutorial is a hands-on introduction to the theory and concepts behind sequential and concurrent Erlang programming, explaining the Erlang syntax, semantics and concurrency model. We conclude with an overview of the error handling mechanisms used to build fault tolerant systems with five nines availability.

The target audience are software developers and engineers with an interest in server side applications and massively concurrent systems.

To learn more about the tutorial, please go here and if you want to book it click here.

The tutorial will take place in the Caesar Room, Imperial Hotel, Russell Square, London WC1B 5BB.

See you there!

January 11, 2010 04:32 PM

19 November 2009: ETC at the 11th Symposium on Trends in Functional Programming

Erlang Training and Consulting Ltd. sponsors the 11th Symposium on Trends in Functional Programming and sits on the programme committee.

The symposium is an international forum for researchers with interests in all aspects of functional programming. It embraces a broad view of current and future trends in functional programming and aspires to be a lively environment for presenting the latest research and applications.

The 11th Symposium on Trends in Functional Programming will be held at the University of Oklahoma, on campus in Norman, Oklahoma, May 17-19, 2010.

January 11, 2010 04:32 PM

12 November 2009: Take our Erlang Certification Survey!

Erlang Training and Consulting Ltd in conjunction with the University of Kent intends to launch a pilot version of an e-certification platform for Erlang in the near future.

To assist in this process, we have launched a survey on Erlang Certification. The objective of the study is to better understand the expectations of users, in order to ensure a well-focused product.

We would appreciate a few minutes of your time in completing this survey. To take the survey, please go to:

http://surveys.erlang-consulting.com/index.php?sid

The results of the survey will be shared with third parties, in publications and as part of research into Certification. When doing so, all results will be completely anonymised and any individual-identifying information removed.

Thank you for your time and cooperation!

January 11, 2010 04:32 PM

28 October 2009: Erlang Web 1.4 has been released!

We are happy to announce the next official release of the Erlang Web framework. The latest 1.4 version is full of new features and enhancements, like integration with EWGI, new template language support: ErlyDTL, Erlang Web Test Suite application bundled, multiple level request dictionary support.

Apart from the above, there are also some changes worth mentioning:
* new wpart introduced: wpart:comment
* embedding default values in wparts from templates (patch submitted by Elena Bobrova)
* top-down template inclusion feature
* experimental ejabberd-flavoured e_hook introduced
* upgraded to Yaws 1.85

Erlang Web 1.4 can be easily downloaded as a tarball from its official website and checked out from the bitbucket repository. The documentation can be found on Erlang Web's wiki page.

We do encourage everyone to try Erlang Web 1.4 out, send us comments, contribute and enjoy the coding!

January 11, 2010 04:32 PM

13 October 2009: Ready for the Erlang courses in London?

Have you always wanted to learn more about Erlang, but never knew how to get started?

We have a special offer, which may encourage you. If you sign up now for a five-day Erlang By Example or Erlang Open Telecom Platform courses in London you will get the Early-Bird Discount of £245 off the standard price! Don't miss out and register now.

     
To see the full list of our scheduled courses, please click here.

January 11, 2010 04:32 PM

Debasish Ghosh

A Case for Orthogonality in Design

In Learning, using and designing command paradigms, John M Carroll introduces the notion of lexical congruence. When you design a language, one of the things that you do is lexicalization of the domain. If we extrapolate the concept to software designs in general, we go through the same process with our domain model. We identify artifacts or lexemes and choose to name them appropriately so that the names are congruent with the semantics of the domain. This notion of lexical congruence is the essence of having a good mnemonics for your domain language. I found the reference to Carroll's work in Testing the principle of orthogonality in language design, which discusses the same issue of organizing your language around an optimal set of orthogonal semantic concepts. This blog post tries to relate the same concepts of orthogonality in designing domain models using the power that the newer languages of today offers.

The complexity of a modeling language depends on the number of lexemes, their congruence with the domain concepts being modeled and the number of ways you can combine them to form higher order lexemes. The more decoupled each of these lower level lexemes are, the easier they are to compose. When you have overlapping concepts being modeled as part of lexemes, the mixing is not easy. You need to squeeze in some special boundary conditions as part of composition logic. Making your concepts independent yet composable makes your design orthogonal.

The first time I came across the concept of orthogonality in design and consciously appreciated the power of unification that it brings on to your model, is through Andrei Alexandrescu's idea of policy based design that he evangelized in his book Modern C++ Design and in the making of the Loki library. You have orthogonal policies that are themselves reusable as independent abstractions. And at the same time you can use the language infrastructure to combine them when composing your higher order model. Consider this C++ example from Andrei's book ..

template
<
  class T,
  template <class> class CheckingPolicy,
  template <class> class ThreadingModel
>
class SmartPtr;


CheckingPolicy enforces constraints that need to be satisfied by the pointee object. The ThreadingModel abstraction defines the concurrency semantics. These two concerns are not related in any way between themselves. But you can use the power of C++ templates to plug in appropriate behaviors of these concerns when composing your own custom type of SmartPtr ..

template SmartPtr<Widget, NoChecking, SingleThreaded>
  WidgetPtr;


This is orthogonal design where you have a minimal set of lexemes to model otherwise unrelated concerns. And use the power of C++ templates to evolve a larger abstraction by composing them together. The policies themselves are independent and can be applied to construct arbitrary families of abstraction.

The crux of the idea is that you have m concepts that you can use with n types. There's no static relationship between the concepts and the types - that's what makes orthogonality an extensible concept. Consider Haskell typeclasses ..

class Eq a where 
  (==) :: a -> a -> Bool


The above typeclass defines the concept of equality. It's parameterized on the type and defines the constraint that the type as to define an equality operator in order to qualify itself as an instance of the Eq typeclass. The actual type is left open which gives the typeclass an unbounded extensibility.

For integers, we can do

instance Eq Integer where 
  x == y =  x `integerEq` y


For floats we can have ..

instance Eq Float where 
  x == y =  x `floatEq` y


We can define Eq even for any custom data type, even recursive types like Tree ..

instance (Eq a) => Eq (Tree a) where 
  Leaf a         == Leaf b          =  a == b
  (Branch l1 r1) == (Branch l2 r2)  =  (l1==l2) && (r1==r2)
  _              == _               =  False


Haskell typeclasses, like C++ templates help implement orthogonality in abstractions through a form of parametric polymorphism. Programming languages offer facilities to promote orthogonal modeling of abstractions. Of course the power varies depending on the power of abstraction that the language itself offers.

Let's consider a real world scenario. We have an abstraction named Address and modeled as a case class in Scala ..

case class Address(no: Int, street: String, 
                   city: String, state: String, zip: String)


There can be many contexts in which you would like to use the Address abstraction. Consider printing of labels for shipping that needs your address to be printed in some specific label format, as per the following trait ..

trait LabelMaker {
  def toLabel: String
}


Note that printing addresses in the form of labels is not one of the primary concerns of your Address abstraction. Hence it makes no sense to model it as one of the methods of the class. It's only required that in some situations we may need to use the Address to print itself in the form of a label as per the specification mandated by LabelMaker.

One other concern is sorting. You may need to have your addresses sorted based on zip code before submitting them to your Printer module for shpping. Sorting may be required in combination with label printing or as well as on its own - these two are orthogonal concerns that should never have any dependence amongst themselves within your abstraction.

Depending on your use case, you can decide to compose your Address abstraction as

case class Address(houseNo: Int, street: String, 
  city: String, state: String, zip: String)
  extends Ordered[Address] with LabelMaker {
  //..
}


which makes your Address abstraction statically coupled with the other two.

Or you may like to make the composition based on individual objects which would keep the base abstraction independent of any static coupling.

val a = new Address(..) with LabelMaker {
  override def toLabel = {
    //..
  }
}


As an alternative you can also choose to implement implicit conversions from Address using Scala views ..

object Address {
  implicit def AddressToLabelMaker(addr: Address) = new LabelMaker {
    def toLabel =
      "%d-%s, %s, %s-%s".format(
        addr.houseNo, addr.street, addr.city, addr.state, addr.zip)
  }
}


Whatever be the implementation, take note of the fact that we are not polluting the basic Address abstraction with the concerns that are orthogonal to it. Our model, which is the design language treats orthogonal concerns as separate lexemes and encourages ways to compose them non invasively by the user.

by Debasish (ghosh.debasish@gmail.com) at January 11, 2010 12:46 PM