The TADS agenda is packed, we have several excellent presentations we’ve not been able to include.  So we’re covering them on the TADS weblog.  We kick-off with Telethings from Thomas Quintana.  We’ll be covering many more as the purpose of the TADS weblog to not only to capture all the cool stuff happening at TADS in November, but also to provide a resource for Telecom Application Developers across all the technologies available to them.

Author: Thomas Quintana, quintana.thomas@gmail.com

Open source is a core component of the Telecom industry.  Problems arise when deploying these technologies to be scalable, robust, and resistant to failure. Telethings is an open source technology whose purpose is to take the science out of deploying telecom infrastructure at any scale.

Using the example of a basic highly available PBX deployment within one data center, see diagram below.  We have not included DNS (Domain Name Server), a disaster recovery solution possibly involving Chef or Puppet, or geo-distribution and we can see how quickly a basic production deployment can start to get complex. This increase in the number of components has an important consequence which is the increased cost of deployment and maintenance. This means we want to consolidate as much infrastructure as possible to keep costs from getting prohibitively high. This notion applies equally from virtualized environments like the cloud to bare metal environments.

The TeleThings Solution

Telethings solves the scalability problems through the following principles:

• “A lightweight container with strong isolation” – Telethings opted to use the lightweight container facilities provided by the Linux kernel (versions 2.6.24 and greater) such as cgroups and namespacing. These technologies allow fine grained resource management and independent world views inside each of the containers. We can treat these containers like we would treat real virtual machines. They can be started, stopped, restarted, cloned, etc.  Integration with these facilities is provided by libvirt-lxc. .
• “A standardized method to dynamically provision new containers” – The DevOps community has been hard at work improving technologies that auto provision IT assets.  The current leading open source projects in this realm are Chef and Puppet with large communities and lots of tools. Therefore, provisioning and configuration of containers is delegated to either of these platforms.
• “An intuitive and consistent method to wire new containers together to provide a targeted service” – Dynamic or Static configuration is returned as JSON objects via HTTP callbacks from the Telethings Broker and it in turn hands it to the Chef or Puppet script for processing. Configuration scripts are usually small and target a specific service. For more complex requirements many small targeted scripts can be executed sequentially to build up complex services. Also configuration scripts are usually provided separate from deployment scripts to allow containers to be reconfigured at runtime using the configuration scripts. Notable exceptions are services that will only be configured only once. In order to reconfigure these services the user must destroy and re-deploy the services using the Telethings Broker.
• “A standardized method to monitor deployed infrastructure” – There are many strategies to monitor the health of infrastructure but Telethings has opted in favor of the SNMP protocol for monitoring and management of networked services. Therefore, Telethings tries to enable SNMP when ever possible. On a side note, the Telethings team is currently investigating alternative methods for monitoring of services that don’t support the SNMP protocol.
• “A strategy to handle automated disaster recovery when infrastructure fails” – The algorithm that chooses what to do when a service fails is pluggable. This enables users to handle failure based on the amount of available resources and any other business or technical constraints. The default strategy is to destroy any containers associated to a particular resource upon failure and have the orchestration service rebuild the service from scratch using the original  configuration parameters.

Telethings is designed to be modular and scalable which is why we had to develop for technologies that facilitated those objectives rather than getting in the way. In order to accomplish this we have broken up the project into pieces and use message passing for inter-process communications with Apache ActiveMQ as the default message broker. Next, in order to keep track of the state the environment is in the broker uses a MongoDB cluster.  An important note to make is that the storage back-end is a pluggable piece in the Telethings platform but the default is MongoDB with many more coming soon.

The Telethings Agent is responsible for managing the Linux Containers in a Telethings Node. The Telething Broker does all the heavy lifting with the assistance of other projects or third party services. It performs orchestration of services on behalf of users to disaster recovery when failure is detected. In the following diagram we can see a little more detail of some of the responsibilities handled by the Telethings Broker.  You can find out more at the Telethings.