Data-first Applications

This is a tutorial/vision page describing the state of the art as I see it today.

Data has become more important than code.

• Collecting it
• Analyzing it
• Storing it

The web plus mobile apps are the user-interface of today's stack. Virtual Reality and Augmented Reality are becoming important quickly as a platform in their own right. Javascript has become more important than Java, both on the server and inside the browser.

Heroku, Amazon Web Services, and similar compute and storage services make it possible for anyone with an idea to write applications that anybody in the world can use, and to take advantage of the data collected by those applications.

I believe that Clojure and Clojurescript (Clojure inside the web browser) provide an opportunity to learn to develop web applications unlike anything that has come before. This is because Clojure is a first-class functional programming language, and because that same language works everywhere.

A programmer can learn Javascript and do similar things, but I believe that Clojure and Clojurescript can take a programmer farther than Javascript can. This is because Clojure is both a first-class functional language and because it is a Lisp, which means that code is data is code is data….

Here is my (current) favorite stack for building and deploying data-first applications. To those who have asked me to share a “way in” to modern software development, this is my list of topics to search on Google.

• Git / Github
• Basic Linux/Unix command line skills
• jitpack.io to share libraries with myself and others
• Clojurescript and Javascript for building single-page web applications
• Hoplon, Javlin, and Castra to create a unified programming environment on the web browser and inside the server using Clojure and Clojurescript.
• A-Frame for virtual reality applications delivered via the web
• Clojure and/or Scala (server)
• Boot and Leiningen for building libraries
• Pouchdb (both inside the browser and inside the server)
• SQL as needed. Postgres is decent as a standalone database. If you really have to scale to big data moving fast, consider Cassandra backed by Amazon S3 on AWS. Use Netflix projects available on Github to wire them together.
• The j2v8 library for integrating the best of Node.js and Java inside a server

Functional programming (FP) helps process data in parallel–either on multiple cores or across a cluster.

That topic is sufficiently deep that it deserves a page of its own.

General notes about JVM development:

• In Java 5, 0 is not always equal to 0
• Code Bloat Thread from E4 mailing list