Book Review: The Rise and Fall of American Growth: The U.S. Standard of Living Since the Civil War by Robert J. Gordon

the-rise-and-fall-of-american-growth-2I read it as an audiobook and it doesn’t make a good one. It’s full of figures, numbers, charts, diagrams, etc which get lost in the audio version. This might cause a bias on my review.

The book feels thoroughly researched. I caught a couple of small errors but most time when I thought “What about this or that” the book answered my objection a couple or paragraphs earlier.

The first two parts cycle between a dry list of numbers and little stories or descriptions that are very interesting. It is, as the title suggest, very American centric but every now and then it compares USA with Europe.

The last part is different, specially the last two or three chapters, which talk about the present and the future. I was heavily entrenched in what this books call techno-optimisim. Techno-optimists believe progress is happening and it will accelerate. AI will solve all problems and destroy all jobs. We also believe robots are the biggest source of unemployment today. This book made me challenge these assumptions. I’m not sure what to believe. Some of the conclusions that you may arrive at from the information here can be very xenophobic/protectionist.

The postscript, America’s Growth Achievement and the Path Ahead is great and concise. I think it should be mandatory reading to be a politician of any kind. Actually, the US should elect Robert J. Gordon as their president.

★★★★☆

Buy The Rise and Fall of American Growth: The U.S. Standard of Living Since the Civil War in USA
Buy The Rise and Fall of American Growth: The U.S. Standard of Living Since the Civil War in UK
Buy The Rise and Fall of American Growth: The U.S. Standard of Living Since the Civil Warin Canada

My take on an AJAX re-frame effect handler

There’s already an AJAX re-frame effect handler provided by Day8, the same guy who made re-frame and there’s nothing wrong with it. From the documentation, this is how you use it:

(re-frame/reg-event-fx ::http-post
  (fn [_world [_ val]]
    {:http-xhrio {:method          :post
                  :uri             "https://httpbin.org/post"
                  :params          data
                  :format          (ajax/json-request-format)
                  :response-format (ajax/json-response-format {:keywords? true})
                  :on-success      [::good-post-result]
                  :on-failure      [::bad-post-result]}}))

That felt a little to verbose for my taste, so I made my own that you use like this:


(re-frame/reg-event-fx ::http-post
  (fn [_world [_ val]]
    {:ajax {:post            "https://httpbin.org/post"
            :params          data
            :on-success      [::good-post-result]
            :on-failure      [::bad-post-result]}}))

If you are familiar with cljs-ajax, you’ll notice the pattern. Day8’s solution uses the raw cljs-ajax API, while I use the higher level one. Day8’s is more versatile, mine is more compact.

This is my solution:

(ns ajax-fx-handler
  (:require [ajax.core :as ajax]
            [re-frame.core :as re-frame]
            [clojure.set :as set]))

(re-frame/reg-fx :ajax
  (fn http-effect [request]
    (let [request-verb (set/intersection #{:get :post :put :patch :delete :head :options :trace :purge}
                                         (set (keys request)))]
      (if (not= 1 (count request-verb))
        (throw (js/Error (str "When specifying an AJAX request, one and only one verb should be specified. Found: " request-verb)))
        (let [request-verb (first request-verb)
              url (get request request-verb)
              request (if (contains? request :on-success)
                        (assoc request :handler #(re-frame/dispatch (conj (:on-success request) %)))
                        request)
              request (if (contains? request :on-failure)
                        (assoc request :error-handler #(re-frame/dispatch (conj (:on-failure request) %)))
                        request)
              request (-> request
                          (dissoc request-verb)
                          (dissoc :on-success)
                          (dissoc :on-failure))
              ajax-fn (cond
                        (= request-verb :get) ajax/GET
                        (= request-verb :post) ajax/POST
                        (= request-verb :put) ajax/PUT
                        (= request-verb :patch) ajax/PATCH
                        (= request-verb :delete) ajax/DELETE
                        (= request-verb :head) ajax/HEAD
                        (= request-verb :options) ajax/OPTIONS
                        (= request-verb :trace) ajax/TRACE
                        (= request-verb :purge) ajax/PURGE)]
          (ajax-fn url request))))))

Feel free to use it, but, is it worth releasing it as a library? does anybody want that?

Book Review: The Obstacle Is the Way: The Timeless Art of Turning Trials into Triumph by Ryan Holiday

81SLYRdcbaLThis book is a sort of different explanation of the concepts of Stoicism. The book claimed the original works by Seneca and others are very accessible but I found The Obstacle is the Way way easier to digest.

This books claims obstacles are a good thing and tries to prove it with many examples of people that achieved great things thanks to their obstacles. I’m not sure I agree. I think there’s a survival bias I that analysis similar to the one the author points to when looking at a list of millionaire college drop outs.

I wish the book would prove things by using data instead of anecdote but I’m giving it four stars because it this book made me think. It made me think about my current obstacles and the attempt of thinking of them as a positive thing made me find new solutions (or reconsider previously discarded solutions). I still thing they are obstacles and that they are bad for me, but they seem more surmountable. That’s no small feat for a book.

★★★★☆

Buy The Obstacle Is the Way: The Timeless Art of Turning Trials into Triumph in USA
Buy The Obstacle Is the Way: The Timeless Art of Turning Trials into Triumph in UK
Buy The Obstacle Is the Way: The Timeless Art of Turning Trials into Triumph in Canada

Pre-signed S3 URLs with Clojure and ClojureScript

Dashman uses S3 for storing the encrypted screenshots. They can go directly from the renderer application to S3 and from there to the displayer application as there’s no need for the server application to ever see them. They are end-to-end encrypted anyway.

My needs in this regard were a bit unique. I have a server component that can generate whatever tokens and signatures are needed and then send it to both applications. The applications are written in ClojureScript, so, essentially, JavaScript, but unlike most direct-to-S3 upload solutions, I don’t have a web site with a form, so, I don’t need to fiddle with CORS or forms.

The two libraries

I found two libraries that could have helped me. The big one is amazonica, a Clojure library to interact with AWS. It looks very complete but when I tried to use the com.amazonaws.services.s3/generate-presigned-url function, I got this error:

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

There are two versions of signatures, version 2 and 4. Version 2 is deprecated, it might or might not work on some data centers but it definitely doesn’t work on all. I refused to believe it at first because I was using very recent version of all libraries, but yup, it was generating version 2 of the signature with signed URLs that look something like this:

https://bucket-name.s3.amazonaws.com/key-name?AWSAccessKeyId=AKI...OA&Expires=1494190327&Signature=ct...3D

Digging further I found this snippet of code:

(System/setProperty SDKGlobalConfiguration/ENFORCE_S3_SIGV4_SYSTEM_PROPERTY "true")

But for me, it made no difference in my case. For the record I reported the issue and it is supposed to work, but it doesn’t for me.

Then there’s s3-beam, that looked promising, but it seems targeted towards forms, not just plain PUT requests from the clients. On top of that, I didn’t want to mess with async, I didn’t need it and it would be over-engineering for my case. Lastly, it uses it’s own custom way of creating the signatures instead of Amazon’s SDK and that code looks like it’s generating the old signatures (sha1 instead of sha256). I ended up not even trying it.

Instead, I decided to do my own custom option, which was not hard and worked straight away. This post is about sharing that code.

Custom solution using the AWS Java SDK

Starting on the server/Clojure side of things, I added the dependency:

[com.amazonaws/aws-java-sdk "1.11.126"]

With only that I was getting this unexpected error when trying to use the AWS SDK:

ERROR com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
ERROR Could not initialize class com.amazonaws.partitions.PartitionsLoader
which I resolved by adding these other dependencies as well (mostly about getting the latest version):
[com.fasterxml.jackson.core/jackson-core "2.8.8"]
[com.fasterxml.jackson.core/jackson-databind "2.8.8"]
[com.fasterxml.jackson.core/jackson-annotations "2.8.8"]

I wrote a function to generate the S3 client:

(defn aws-s3-client []
  (let [region (Regions/fromName (config/env :aws-s3-region))
        credentials (BasicAWSCredentials. (config/env :aws-access-key-id)
                                          (config/env :aws-secret-access-key))]
    (-> (AmazonS3ClientBuilder/standard)
        (.withRegion region)
        (.withCredentials (AWSStaticCredentialsProvider. credentials))
        .build)))

config/env just fetches configuration values, powered by cprop and mount. This is just standard stuff from Luminus. With

(defn- pre-signed-url [key & [verb]]
  (let [expiration (tc/to-date (-> 24 t/hours t/from-now))
        generate-presigned-url-request (doto (GeneratePresignedUrlRequest. (config/env :aws-s3-bucket-name) key)
                                         (.setMethod (if (= verb :put)
                                                       HttpMethod/PUT
                                                       HttpMethod/GET))
                                         (.setExpiration expiration))]
    (.toString (.generatePresignedUrl (aws-s3-client) generate-presigned-url-request))))

The function generates either a URL for reading, :get, or for writing, :put, depending on the parameter verb. key is just the path part of the URL (remember, S3 is actually a key/value storage). All URLs will last for 24hs, something you can tweak as well.

And that’s it, generating the URL is done, now we have to use it.

Reading and writing from ClojureScript

If you have a form, then, you might need to deal with CORS and other issues. I suggest you look at s3-beam at least for inspiration. For me, it was just a matter of generating the requests from the ClojureScript app, which uses cljs-ajax. Reading was straightforward enough:

(ajax/GET get-url
          {:handler (fn [value]
                      (do-something value))})

although the handler might have to deal with encoding issues. Writing is similar, but again, you may need to fiddle with encoding issues. Since what I was sending is a JSON payload (an encrypted screenshot), this worked for me:

(ajax/PUT put-url 
          {:format :json
           :params payload})

If you are uploading binary data, I recommend you play with :format :raw and the :body attribute.

How to work with a private library in ClojureScript

Dashman is composed of many components  including three desktop apps written in ClojureScript using Electron that share code through a private library (as in, it’s not open source).

To have continuous integration and a set up that is easy to boostrap, that library should be deployed to a private repository. I achieved that using the plug in s3-wagon-private so that the project.clj of the common library looks like this:

:repositories [["private" {:url           "s3p://s3-bucket-for-my-project/releases/"
                           :username      "not-telling-you"
                           :passphrase    "not-telling-you-either"
                           :sign-releases false}]]

:plugins [;...
          [s3-wagon-private "1.3.0"]]

You should never have production or even staging credentials on your source code repository; but those credentials are neither. They allow to compile the project so, pretty much everybody that has access to the source code also needs access to that private repository. Having them in project.clj simplifies CI, on-boarding developers, etc.

One you have that you can deploy the library by executing:

lein deploy private

The project of the apps using the common library look similar:

:repositories [["private" {:url           "s3p://s3-bucket-for-my-project/releases/"
                           :username      "not-telling-you"
                           :passphrase    "not-telling-you-either"
                           :sign-releases false}]]
:dependencies [;...
               [tech.dashman/clientcommon "0.1.0-SNAPSHOT"]]

:plugins [;...
          [s3-wagon-private "1.3.0"]]

Yes… very similar.

The next issue is compiling and re-loading code. You could modify the common library, install it, re-compile the apps, and re-run the apps. And you could do it while wearing a mullet and watching MacGyver (the original one). This is not the 80s! You want code reloading and all the goodies.

If you are using Clojure, you should look into checkouts and maybe lein-checkout. In ClojureScript you can just add the path of the source code to your project even if it’s outside the project, with one gotcha. You might be used to a config like this:

:cljsbuild {:builds {:app {:source-paths ["src/app"]}}}

which would imply this might work:

:cljsbuild {:builds {:app {:source-paths ["src/app" "../clientcommon/src"]}}}

It might on non-Windows platforms, on Windows it doesn’t. On Windows you want this:

:cljsbuild {:builds {:app {:source-paths ["src/app" "..\\clientcommon\\src"]}}}

Since you should probably never commit that anyway, it’s not a big deal.

ClojureScript debouncer

I’m developing Dashman in ClojureScript (and Clojure) and as I was developing the cookie handler I found sites that constantly modify the cookies. As in, many times per second. Obviously I don’t want to encrypt the cookies and send them to the server that often, but even without taking that into account, just storing them to keep track of them in an atom, as in:

(reset! cookies new-cookies)

was slowing down the app considerably. I found myself needing to debounce so many calls as there’s no value in the middle calls, only the last one. Let’s backtrack a little bit here.

The term to debounce comes from electronics:

To remove the small ripple of current that forms when a mechanical switch is pushed in an electrical circuit and makes a series of short contacts.

In programming it refers to a signal that gets emitted often and you want to process it but you don’t care about processing every intermediate value. To tie into electronics imagine you have a program that receives a message every time the temperature changes and the message contains all the temperature changes for the day. Clearly each message doesn’t have to be processed, only one, only the latest.

Here’s where a software debouncer can help. It’s a function that gets a function and returns another function. When you call the resulting function 10 times, it will only call the original function once, with the last value it received.

I found this debouncer written by David Nolan: https://gist.github.com/swannodette/5888989 and in the comments you can start to see that writing a debouncer might not be a trivial as you may think. One of the comments point to this other version: https://gist.github.com/scttnlsn/9744501. The comment implies this is a more correct version but I haven’t verified it myself. In that last page you can see someone found a bug and posted an improved version.

For me, when a supposedly simple code snippet generates so many comments and versions, it’s better to find a library that implements it than to copy and paste and never maintain it again. Unfortunately, I couldn’t find a ClojureScript debouncer library but I got pointed to goog.function.debounce which you can use this way;

(ns whatever
  (:require [goog.functions]))

(def debounced-println (goog.functions.debounce println))

Book Review: The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change by Stephen R. Covey

The 7 Habits Of Highly Effective People 3I think I had my expectations wrong for this book. I was expecting it to focus on business and professional growth when most of its focus is on relationships and families. And when it addresses the professional life, it’s from the point of view of a manager/leader of a big company.

It has some interesting concepts such as the emotional account that I think if everybody followed them it would make the world a happier place. But I strongly disagree that there’s a correlation between following those values and one being more effective. For example, the book asserts that if you treat people respectfully, they’ll respect you back. This is not true. This is not how the world works. This is not how the brain work either and some of the facts about the brain that this book cite have been proved false.

Lastly but not least, the book gets awfully preachy, jumping into religion like if that was fact or proof of anything.

All in all I think the book pretends to be scientific but it’s very dogmatic. I’d recommend avoiding it.

★☆☆☆☆

Buy The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change in USA
Buy The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change in UK
Buy The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change in Canada