Clojurescript (CLJS) – ReactJS, Re-Frame, and Reagent

I’ve been using Re-frame pretty extensively these last few weeks as we begin to work on the front end of our horrendously stylized project called Pipeline.  It has been pretty tough to get the hang of but after being stubborn enough not to change my mind once going down this road I’m really starting to come around … and dare I say, actually understand how to make it work!

What’s the Big Idea?

There are a few components to what I’m talking about that I’ll go over below.  The big idea though is that web applications today are a big mess of javascript, html, and CSS that is somehow stitched together.  Most of todays web applications aspire to be Single Page Applications (or SPA if you are into that whole brevity thing).  Several frameworks have emerged with the (arguably) popular ones being Angular.js (from Google) and React.js (from Facebook).  The big picture though is:  How do you create a great Web front end to your application?  These frameworks typically call the APIs of a web backend which (hopefully) would be the same backend that a mobile application would call.

Clojurescript

Clojurescript is a LISP language.  It’s basically the same language/syntax as Clojure but where as Clojure compiles down to Java, Clojurescript compiles down to Javascript.  Why would you want that?  Seems like overhead.  Well, the claim is that the Google closure compiler (confusing name) is much more efficient and can make it faster.

What is interesting to me as I go down this route is that most of the blog posts are from 2014 and earlier and I can’t tell if I’m pursuing a path that is no longer cool or what.  This is surprising to me because with the greater popularity of React these days this seems like a great path.

The downside to clojurescript is the stack traces are difficult to comb through and understanding the whole immutability thing can be a struggle.

React

React is Facebook’s Javascript library for creating user interfaces.  The idea is that there is a virtual dom and it listens to changes in data (binds to data) then will respond immediately to the changes.  It’s funny that after using the library through Reagent I still don’t really understand React.  To me its just the data changes and the web page is immediately updated.

It should be noted that React isn’t a full on featured library like Angular so some people say its unfair to compare them.  But I tend to like this blog’s argument of why you can.  And in my limited knowledge of things in the world am easily persuaded by his post.  (Or should I say, enjoy that he has validated my decision).  I especially like the “unix philosophy” argument that takes the position of: “Your tool should do one thing very well”.

Reagent

So you have React and you have Clojurescript and you want to make awesome happen.  So naturally people create frameworks to make this happen for you.  Om, Rum, and some others emerge and I think are still out there.  I settled on Reagent though cause it seemed pretty simple and easier for me to wrap my head around.  Reagent uses the hiccup syntax so it gets rid of having to do html tags and makes it easier to see.  When I used Ruby on Rails I would use HAML which was the same concept.

The result is that now when you are writing the application everything is in one language!  There’s no sprinking of HTML and not mixing with Javascript, its all just Clojurescript!  This is the thing I most love about it.  Its more akin to me like designing an iOS application in swift where there is one language to do it all.

The other powerful aspect of Reagent is that it is functional.  Functional Reactive Programming (or FRP) means that interfaces respond to the data.  Or, “everything is data derived”.  There’s no ngThis or ngThat, it’s just functional stuff.

The issue with Reagent (or lack of design feature) is that it is silent on how to store data and interact with data (client side) as well as serverside.  This is where the grand daddy of them all comes into play:  Re-Frame

Re-Frame

I started with examining the README file on the Re-Frame project and was immediately blown away.  Yes!  This is the most epic readme of all projects.  The examples and docs were pretty clear and quite glorious.  I loved the opinionated nature of it and the humor.  (not the best reason to pick a framework, but not bad!)

Re-frame is a framework that works with Reagent to create the idea of a clientside database.  (Or a database that lives in the browswer).  It makes things fast and helps with state changes.  It has the idea of subscribing to changes or dispatching changes to the database realizing that everything can be asyncronous but everything responds to changes in the data.  Data all the way down.

An Example

I put an example on github that you can see if you want to run for yourself.

The idea is that you have a list of names and you want to filter them in the search bar.  Here’s the main entry code that is called from the main reagent code that initializes this home page.

I have several bootstrap tags in there that I took out of the code.  The tricky thing is to notice the reagent r/atom which is the data that changes.  This code doesn’t have any re-frame code to it.  Basically it displays a form.  Notice the code at the bottom to call display-names.  That code is where the filtering happens.  When the input is changed, the search-string changes.  You never have to re-call display names as its always called and responds to changes in data.  So great!

Above is the display names code that takes a list of names (defined outside the function in the def (or variable ) called “names”.  It goes through and prints a name if it matches the filter.

Leadership

I’ve had the pleasure of meeting Adrian Cockcroft of Battery Ventures a few brief moments over the course of my career.  The first time was when he worked for eBay and I worked for IBM and we were doing a GPFS proof of concept.  He was a really nice person (and still is!) and introduced me to LinkedIn.  In fact he was my first connection!  The next thing I knew, years later he had done incredible things at Netflix and continues to do so at Battery Ventures.  He’s said a lot of cool things through the years, but the one story that has been the most impactful to me was his story of the reaction of a Fortune 100 CTO that approached him after he gave a talk.

Adrian’s talk I imagine was about all the cool DevOps things Netflix was doing.  The CTO came up to him afterwards and his reaction was:  “But Netflix has a superstar development team, we don’t!”

Adrian’s response was classic:  “Well, we hired these guys from you”.

The lesson I take from this is the impact that great leadership can have and conversely how poor leadership can squander opportunities and waste millions of dollars.

I’ve been on many teams in my career.  I’ve been really lucky to have been on some of the best.  The various teams I’m a part of now at Cisco are first class.  But there have been times when I’ve thought:  If we only had a better team or people that understood more about X or Y, then just imagine what we can do.

Given what Adrian’s story illustrates, I am changing my thinking to believe that I am already on the best team where ever I go.  The potential for greatness that is there is unlimited.  Everyone I work with really wants to do a good job.  We’re driven, passionate and want to be the best.  When there are issues, leadership from within the team can surmount those obstacles and make the team better.

I imagine somewhere there are executives in big companies trying to figure out a market they haven’t been able to crack.  And they might be thinking:  If only we had top developers who could get us there.  Then they might think: Maybe we should acquire a company that has top developers and they can get us there.

Well, even though they’re not reading this, I would say to them:  You already have some of the best people working for you and all the resources at your disposal.  There’s actually another problem here, and it isn’t your people.

And if you disagree with decisions that have been made, you can lead from within and be the leader you want to see.  Even if the only one you’re leading is yourself.

Drone Secrets

I was really happy to see the Drone Secret’s page describe how to put secrets in a .drone.yml file.  Checking passwords into repositories is a big no-no.

Still, there were some clarity in the docs I would have liked.  Here’s step by step.

1. Install drone

Yep. This is the mac client.  I did the manual way

2.  Set environment variables

You have Drone up, set the following in your .bash_profile or .login

The DRONE_TOKEN you can get by logging into drone and clicking on your profile.  The settings area has that.

3.  Create the secrets.yml file as shown in the docs.

4.  Convert and check in!

5.  Secrets can be accessed in the .drone.yml file with the $${VARIABLE}

The example below shows the QUAY_PASSWD variable.

 

Drone on CoreOS for CI/CD

I’m working on a moving my CI/CD platform to Drone.  As usual, I’m behind a corporate firewall

1. Install CoreOS

This is just a standard CoreOS image on OpenStack

2. Configure Proxy

Now update to get latest settings so Docker works

3. Get Drone

4. Register application in your github account.

Since I’m using an enterprise version of github, I went into the organization and created an oath account

githuboath

5. Create a configuration file with the environment

I created /vol/dronerc.  My /vol directory is a persistent storage volume mounted to my vm.  The contents are as follows:

6. Start up Drone

Now we can go to our page and open up our Drone app.

You can see that I mapped the default 8000 port to port 80 so that I can access it directly from my server, which is hosted internally at 10.93.234.142

Jumphost SSH config

Oh, you have a jumphost? And then you want to get to another server that is behind that server on a private network?  If both servers are configured with the same key this can be accomplished by putting an SSH config file entry like the one below:

To get to the amster server I now just run:

It runs through the jump.amster server and instant access.

Cassandra Startup with Docker and Golang

I use Docker:

 

(Notice that using boot2docker or docker machine on virtual box doesn’t work very well because the directories are created under different user name.  You have two choices:  Don’t mount persistent directories with boot2docker or log into the virtual box guest (docker-machine ssh dev) and launch the container there.)

Make sure its up:

 

Now lets create a Keyspace (or Database in SQL terms).  Connect to it by connecting to cassandra where the 9042 port is mapped:

Here we see that port 9042 is mapped to 32769.  To connect we run:

Now we can create the keyspace:

 

Now let’s use it with a Go program.

Our code is pretty simple:

If that worked then you created an entry and go data back.  You are now off to the races writing golang code to a cassandra database.

TODO:

  1. Scale out to multiple cassandra servers
  2. Show how this works in mantil.io

Ethereum Contracts

I’ve been working with Eris to manipulate “smart contracts” (smart contracts are neither smart, nor are they contracts.  They’re just code written to the blockchain that stores values that can be changed with the right permissions).

Eris has a pretty good ‘eris-by example’ script that shows how you can do a lot of it with curl.  My object was to not use the Eris tools but make it a bit more generic for use with any Ethereum contract.

I’ll gloss over creating the contract, but essentially, I just created a quick contract called ‘echo’ and compiled it into the solidity compiler. (Yes I know it moved by the new link didn’t work for me )

My code is pretty simple and borrowed from the millions of other echo “hello world” like contracts where it stores a string and an integer:

After submitting the transaction I get addresses as to where it was submitted.  I’ll gloss over that detail as I’m interested in the other part of this and that is how to interact with the data.

The solidity compiler shows that when it compiles the functions to javascript byte code it creates hashes of the function names themselves.  The documentation shows that to call a function you do it by calling its name ‘derived as the first 4 bytes of the Keccak hash of the ASCII form of the signature’.  Luckily the solidity compiler gives this value to you.

Now you just need to find the address of where the function lives.

In my app I can now call:

Let’s break this down.  The from Address is one of the validators that I created.  The ‘toAddress’ is the address of where the contract lives.  To get the value amount from this we found from the solidity compiler that the function was called by entering in ‘d321fe29’.  That then returns the following:

Our value is encoded in hex, and everyone knows that 64 is actually just 100.

We can do the same thing to get the string value calling the get_s function:

Using the above command but subsituting in 75d74f39 we get the following:

To convert the returned hex into string we look at the output.  The first 32 bit string shows a 2.  This means that the data comprises the next 2 32 byte strings.  Decoding the last part:

Nice message.

Now how to change this value?  If we had a set_amount function we would just add the value after the function.  Since we only have a set string we have to do it dynamically.  In this case figure out what our string will be then pad it with enough 0s.  Strings are ‘dynamic types’ as specified in the documentation.  So we have to add a few more fields.  First the function:

Let’s first make a string:

This will fit in one 64 byte string so we add the extra 0s to the end and add the 64 byte string to tell us the offset for it.  The argument to the function then looks as follows:

Adding this to the one command:

The output will have changed nothing.  Why?  From the README we read that:

” It is possible to “query” contracts using the /call endpoint.  Such queries are only “simulated calls”, in that there is no transaction (or signature) required, and hence they have no effect on the blockchain state.”

So to change this for reals we need to actually sign our call.  I tried several methods at present but eris only returns garbage.  For example here’s a command I ran:

The output was garbled guck.  At this point I’m stuck on this effort but thought I would at least show where I am.  Using the JSON RPC is perhaps too low level and ideally you would use the Eris javascript library (eris-contracts)  My hope instead was to use the RPC API to accomplish this.  Perhaps more to come!

 

 

Jenkins Spark integration

You can make Jenkins publish messages to spark after or during builds.  Here’s how we do this:

1. Get the spark room ID that you want to use:

This will give you a list of rooms.  From here, you can find the room you want and get the ID of the room: Something like: Y2lzY29zcGFtxuovL3VzL1JPT00vOWRhMjY1MDAtOWY2Zi0xMWU1LTg0ODQtNzczOTMxZTUxMGE3

2.  In jenkins we can now notify the room of a build before the build or after the build by using the execute command plugin.  Putting a simple curl command like the below will create the command necessary to notify the spark room:

Just substitute in the roomId and Authorization token.  You can also create a person using a different email account to make a ‘jenkins user’.

Installing Eris

Eris Industries is another provider of blockchain services and smart contracts.  I had a few issues installing it on Ubuntu even though I followed the guide, so I wanted to outline here how I did it.  The doc will be somewhat terse, but will have all the commands I ran.

Base Operating System

I’m running mine on Ubuntu 14.04.  I did this on an Internal OpenStack cloud provided by Cisco. I used 4 servers and provided them with a floating IP address.  I should have done an ansible playbook for this and will do this in the future.

Installation Steps

Update the OS or nothing will work!

Install Docker

Update the sources list create

The contents should be:

Next run the following:

Run

To make sure that it works.  Lastly, make sure the current user can control  docker by adding him to the docker group:

Install Go (Golang)

You’ll now need to add the following to your ~/.profile file at the end:

You may need to log out and log back in to make sure that the docker group is set for the next step.

Install Eris

This is done by issuing the following commands

If you had trouble with this step, make sure that you can run docker with the current user (e.g: no sudo required, see last part of Install Docker section above).  If you have problems with GOPATH, make sure it is set and exported as shown in previous section.

Now you can start rolling your chains!

 

Jenkins running in Docker image behind firewall

When you run jenkins behind a firewall you need it to get out.  You’ll have to set up proxies to let this happen.  Here’s how I make it run:

First, make coreos able to go out of your network:

Next edit http-proxy.conf and make it look like:

This will allow docker to get outside the firewall.

Next restart docker:

Now, on to Jenkins.  Grab Jenkins:

Make a persistent directory to store settings (should be a persistent volume mount)

Now run the container with the following JAVA_OPTS flag as documented here.

You’ll obviously want to use your proxy server instead of mine!

You should now be able to install all the plugins you need!  Hurray!