Java Async IO Package

We need Async IO Package for sheer performance and scalability!
Performance and scalability are key attributes of the IO system for IO-intensive applications. IO-intensive applications are typically, although not exclusively, server-side applications. Server-side applications are characterized by the need to handle many network connections to many clients and also by the need to access many files to serve requests from those clients. The existing standard Java facilities for handling network connections and files do not serve the needs of server-side applications adequately. The java.io and java.net packages provide synchronous IO capabilities, which require a one-thread-per-IO-connection style of design, which limits scalability since running thousands of threads on a server imposes significant overhead on the operating system. The New IO package, java.nio, addresses the scalability issue of the one-thread-per-IO-connection design, but the New IO select() mechanism limits performance.

SQL Vs NoSQL

The advantage of a relational database is the ability to relate and index information. Most key-value systems don't provide that.

Does switching to nosql really make sense for the intended use case?

You have kind of missed the point. The point is, you don't have an index. You don't have a centralized list of records, or the ability to relate it together in any easy way. What makes nosql key-value stores so quick is that you store and retrieve what you need in a name-based approach. You need that blurb on someone's profile page? Just go fetch it. No need to maintain a table with everything in it. This being said, NoSQL has a number of novel structure which make many usecases trivially easy, e.g. Redis is a data-structure oriented DB well-suited to rapidly building anything with queues. MongoDB is a freeform document database which stores documents as JSON.

Not everything really needs to be tabular.

There's advantages and disadvantages. Sometimes using a mix of both can also make sense. SQL for most, and something along the lines of CouchDB for random things that have no need to be clogging up an SQL table.

You can liken a key-value system to making an SQL table with two columns, a unique key and a value. This is quite fast. You have no need to do any relations or correlations or collation of data. Just find the value and return it. This is an oversimplification, NoSQL databases do have a lot of nifty functions beyond simple K,V stores.

You'll find a simple K,V store is also fast in SQL databases. I've used it in place of actual key-value systems before NoSQL databases matured a bit.

I do not think scientific data is well suited to a nosql implementation, but if you look at HBase, it may well suit a scientist's needs.

The efficiency comes from the following areas:

1. The database has far fewer functions: there is no concept of a join and lessened or absent transactional integrity requirements. Less function means less work means faster, on the server side at least.

2. Another design principle is that the data store lives in a cloud of servers so your request may have multiple respondents. These systems also claim the multi-server system improves fault tolerance through replication.

ActiveMQ Java Example

 private static ActiveMQConnectionFactory connectionFactory;
      private static Connection connection;
      private static Session session;
      private static Destination destination;
      private static boolean transacted = false;
 
      public static void main(String[] args) throws Exception {
          BrokerService broker = new BrokerService();
          broker.setUseJmx(true);
          broker.addConnector("tcp://localhost:61616");
          broker.start();
 
          setUp();
          createProducerAndSendAMessage();
          System.out.println("Simulating a huge network delay :)");
          Thread.sleep(4000);
          createConsumerAndReceiveAMessage();
 
          //TODO: Find out how to get rid of the exceptions thrown when stopping the broker
          broker.stop();
      }
 
      private static void setUp() throws JMSException {
          connectionFactory = new ActiveMQConnectionFactory(
                  ActiveMQConnection.DEFAULT_USER,
                  ActiveMQConnection.DEFAULT_PASSWORD,
                  ActiveMQConnection.DEFAULT_BROKER_URL);
          connection = connectionFactory.createConnection();
          connection.start();
          session = connection.createSession(transacted, Session.AUTO_ACKNOWLEDGE);
          destination = session.createQueue("mmy first active mq queue");
      }
 
      private static void createProducerAndSendAMessage() throws JMSException {
          MessageProducer producer = session.createProducer(destination);
          producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
          TextMessage message = session.createTextMessage("Hello World!");
          System.out.println("Sending message: " + message.getText());
          producer.send(message);
      }
 
      private static void createConsumerAndReceiveAMessage() throws JMSException, InterruptedException {
          connection = connectionFactory.createConnection();
          connection.start();
          MessageConsumer consumer = session.createConsumer(destination);
          MyConsumer myConsumer = new MyConsumer();
          connection.setExceptionListener(myConsumer);
          consumer.setMessageListener(myConsumer);
      }
 
      private static class MyConsumer implements MessageListener, ExceptionListener {
 
          synchronized public void onException(JMSException ex) {
              System.out.println("JMS Exception occured.  Shutting down client.");
              System.exit(1);
          }
 
          public void onMessage(Message message) {
              if (message instanceof TextMessage) {
                  TextMessage textMessage = (TextMessage) message;
                  try {
                      System.out.println("Received message: " + textMessage.getText());
                  } catch (JMSException ex) {
                      System.out.println("Error reading message: " + ex);
                  }
              } else  {
                  System.out.println("Received: " + message);
              }
          }
      }

ActiveMQ & Open JMS

ActiveMQ is an open source, Apache 2.0 licensed Message Broker and JMS 1.1 implementation and Enterprise Integration Patterns provider which integrates seamlessly into Geronimo, light weight containers and any Java application.
ActiveMQ provides bridging functionality to other JMS providers that implement the JMS 1.0.2 and above specification. 

A JMS bridge can be co-located with an ActiveMQ broker or run remotely. In order to support JMS 1.0.2 there is separation between Queues and Topics.

Features:

• Supports a variety of Cross Language Clients and Protocols from Java, C, C++, C#, Ruby, Perl, Python, PHP

• OpenWire (cross language Wire Protocol to allow native access to ActiveMQ) for high performance clients in Java, C, C++, C#

• Stomp support so that clients can be written easily in C, Ruby, Perl, Python, PHP, ActionScript/Flash, Smalltalk to talk to ActiveMQ as well as any other popular Message Broker

• full support for the Enterprise Integration Patterns both in the JMS client and the Message Broker

• Supports many advanced features such as Message Groups, Virtual Destinations, Wildcards and Composite Destinations

• Fully supports JMS 1.1 and J2EE 1.4 with support for transient, persistent, transactional and XA messaging

• Spring Support so that ActiveMQ can be easily embedded into Spring applications and configured using Spring's XML configuration mechanism

• Tested inside popular J2EE servers such as Geronimo, JBoss 4, GlassFish and WebLogic

• Includes JCA 1.5 resource adaptors for inbound & outbound messaging so that ActiveMQ should auto-deploy in any J2EE 1.4 compliant server

• Supports pluggable transport protocols such as in-VM, TCP, SSL, NIO, UDP, multicast, JGroups and JXTA transports

• Supports very fast persistence using JDBC along with a high performance journal

• Designed for high performance clustering, client-server, peer based communication

• REST API to provide technology agnostic and language neutral web based API to messaging

• Ajax to support web streaming support to web browsers using pure DHTML, allowing web browsers to be part of the messaging fabric

• CXF and Axis Support so that ActiveMQ can be easily dropped into either of these web service stacks to provide reliable messaging

• Can be used as an in memory JMS provider, ideal for unit testing JMS Can be used as an in memory JMS provider, ideal for unit testing JMS

OpenJMS is an open source implementation of Sun Microsystems's Java Message Service API 1.1 Specification

Features:

• Point-to-Point and publish-subscribe messaging models

• Guaranteed delivery of messages

• Synchronous and asynchronous message delivery

• Persistence using JDBC

• Local transactions

• Message filtering using SQL92-like selectors

• Authentication

• Administration GUI

• XML-based configuration files

• In-memory and database garbage collection

• Automatic client disconnection detection

• Integrates with Servlet containers such as Jakarta Tomcat

• Support for TCP, RMI, HTTP and SSL protocol stacks

• Support for large numbers of destinations and subscribers

Rapid development

In today's cloud environment, where multi-tier architecture is a must, one has to optimize code between both internal & interactive layers.

Most successful software products are those which invest on software infrastructure development such that 90% of development & enhancement tasks are automated and there is not much maintanance cost in the long run.

Here I would like to come up with a strategy which caters to both mobile & desktop devices:

1. Middle-layer that is lean & configurable. All business logic goes here.

2. Use a relational database to initially build the schema. Once the database grows and can't scale, should have the ability to move database tables into file or some other high performance i/o. The infrastructure design should be such that it should facilitate for migration from database to file storage without affecting the middle-layer.

3. Provide database transactions that will be processed reliably. Should confirm to ACID (atomicity, consistency, isolation, durability) properties.

4. Supporting high-performance api to convert binary data into xml, json or any other data format depending on end-user's device (mobile, desktop)

5. A basic CRUD (Create, Read, Update and Delete) framework that an architect should provide for building an enterprise application. Ensure all basic validations are taken care of at the middle-layer. Should support creation of UI code automatically for all targeted devices.

6. Naming conventions should be strictly followed for better readability & maintainability of the code.

7. Middle layer should be fault tolerant and also be able to handle transactions normally in such a way that even if front-end passes the data without validating it (if javascript is not supported) it should still work normally.

7. JUnit for Java, Jasmine for javascript should be used.

8. Use the most compressed data format where ever possible. With javascript now coming up with typed arrays ensure you don't hog your network with redundant data (https://developer.mozilla.org/en/javascript_typed_arrays). Don't use propreitary data formats.

Issue in debugging Tomcat 6 Remote Java application on Eclipse 3.4.2

This is a very simple task. But people get stuck with a timeout issue while the debugger tries to connect to the java vm. Hence have posted this...

create a bat file with below content in your tomcat/bin folder. use this to start your tomcat server.
rem (if it doesn't connect, try 8000)
set JPDA_ADDRESS=8787
set JPDA_TRANSPORT=dt_socket
catalina.bat jpda start

In eclipse go to debug configuration & select remote java application. use localhost if server is local, & set the appropriate port (8787 or 8000 whicever works)

jquery vs dojo vs extjs

When building web 2.0 applications there is always this confusion about which is the best library to use.

After a lot of research on jquery, dojo and extjs, here is my thoughts:

1. I recommend ExtJs if you don't want to spend time & energy in fixing bugs & extending components. Also ExtJs gives components for mobile. Else jQuery. jQuery is free, open source software, dual-licensed under the MIT License and the GNU General Public License, Version 2.[4] jQuery's syntax is designed to make it easier to navigate a document, select DOM elements, create animations, handle events, and develop Ajax applications. Used by over 31% of the 10,000 most visited websites, jQuery is the most popular JavaScript library in use today.

2. Ext includes interoperability with jQuery and Prototype

3. Dojo can be used in javascript-based Adobe AIR applications. It has been modified to meet AIR's security requirements.

Javascript code is difficult to maintain compared to Adobe Flex. Hence I don't prefer to modify most of the components. I prefer to make the most of any javascript library that is open source and has the right component for my application.