Category: Technical

Book Review: Foundation Licence Now by Alan Betts

9781872309804_9c3c0da200c92b4703ae40246d591184I wished someone handed me this book years ago, when I moved to the UK, because I would have gotten my foundation amateur radio licence much, much sooner if I knew how simple it was. I have a similarly leveled licence from Argentina (LU5ARC) and it was much harder to obtain.

I found the structure of the book a bit erratic. Some chapters are formatted much different than others. I didn’t find this to be a problem though, just surprising. The content is generally covered well and I think people with no knowledge of electronics or radio could read it and understand it all.

I guess you could say the proof is in the pudding and whether I’ll pass the test, but being that I already took ham radio courses and I’m an electronic technician, me passing the test is no proof of the quality and usefulness of this book.

★★★☆☆

Buy Foundation Licence Now in USA
Buy Foundation Licence Now in UK
Buy Foundation Licence Now in Canada

Advertisements

Book Review: Getting Started In Amateur Radio by Steve Nichols

36282097I’m not an expert but I’m also not a beginner to ham radio (I’m LU5ARC in Argentina and I’ll be working on getting my UK license next month) and this book opened my eyes to a lot of new cool things to do in the world of amateur radio. I marked a lot of things here to explore once I get my license.

This book even has a section on antennas that give you a good idea of the sizes and types of arenas you could use for different bands.

On the negative, this book has quite a few grammatical errors and typos, which I don’t mind, but it also contains quite a bit of jargon that is never explained. A bit I knew already because of my limited experience with radio, a bit I had to look it up.

★★★★☆

Buy Getting Started In Amateur Radio in USA
Buy Getting Started In Amateur Radio in the UK
Buy Getting Started In Amateur Radio in Canada

An important lesson in consumer protection (for small businesses owners)

I recently bought a desktop computer from Dell and by the time it arrived I was wondering if it was powerful enough, so, I took it for a spin and found out that my suspicious were correct. It was not powerful enough for my needs. We’ll, no big deal, all I had to do was give them a call to arrange for a return and a refund. Dell told me I had 14 days to return it.

When I started the process they asked me if I bought it as an individual or a business. Well, it was for business and it was paid by my business. I didn’t know it at that point, but that meant that the European Union laws for consumer protection didn’t apply.

The EU has a cool off period on online purchases of 14 days. Within those 14 days you can return an item without having a reason. This is for people, not companies. I was exempt of that protection and the terms of service of Dell is: no returns. I got so used to be able to return whatever I want, that this really surprised me. It was surreal.

Did you notice that Dell return policy is the absolute minimum that the EU requires? 14 days for consumers, no return for businesses. I thought most companies were accepting returns because keeping customers happy it’s good for business. Certainly some are, for example, Amazon never refused a return no matter whether it was a business or personal purchase.

This made me wonder: how many other companies look good because the EU is making them and not because they are good. What will happen when the UK leaves the EU? I bet many of these battles will have to be fought again, some will be lost, some will be won.

After a lot of whining and protesting, Dell accepted to take the computer back, but from now on I’ll think twice about buying things from Dell. Even when buying Dell products.

For example, for my new workstation I need a monitor and my research pointed me to one that Dell offers. If you I went to Dell’s web site, and clicked “For Work”, the monitor was 20% cheater than if I clicked “For Home”. Maybe that’s the cost of accepting returns, maybe it’s just dynamic pricing. Thankfully I found the monitor on Amazon, cheaper than the cheapest price Dell was charging, on prime, so delivery was even faster. I obviously ordered them on Amazon.

Picture by www.cafecredit.com

Book Review: Spring REST by Balaji Varanasi, Sudha Belida

9781484208243If you need to build a RESTful API with Java, this book is the way to go. Short and sweet, it covers most of what you need without too much faff.

I particularly liked the chapter on errors. It has quite a bit of code that I copied into my project as a starting point.

When it came to testing, I liked that it acknowledged the need to do integration testing, that is, testing the API with a real database backend. I personally don’t buy the mock everything custom of the Java world as bugs can hide in the integration and these days it’s so easy and fast to test against a real database that I don’t see any value in mocking it out (if you do, the book covers that too).

It has a small section on writing clients using RestTemplate which I wish it was bigger but I understand my needs in this regard are not that common. Particularly, there was no mention on how to deal with the validation errors scheme introduced earlier when using RestTemplate.

All in all, I like this book. For me, computer books should either be short or comprehensive (not padded) and this one is a good example of a good short to-the-point book.

★★★★☆

Buy Spring REST in USA
Buy Spring REST in UK
Buy Spring REST in Canada

Book Review: Beginning Hibernate by Jeff Linwood, Dave Minter

Begining HibernateSweet introduction to Hibernate. I can’t believe it doesn’t cover migrations (and I’m sure some people will point out migrations it’s not part of Hibernate, but without it, there’s no good way to maintain a production database).

The book is on the short side for a computer book and that’s a huge plus. Even then, if you are going to do Spring Boot, like I am, there’s quite a few sections of the book you can skip.

★★★☆☆

Buy Beginning Hibernate: For Hibernate 5 in USA
Buy Beginning Hibernate: For Hibernate 5 in United Kingdom
Buy Beginning Hibernate: For Hibernate 5 in Canada

Using Oracle’s JDK in CircleCI (with Gradle)

I’m evaluating re-writing Dashman in Java (instead of Electron). It’s too early to tell, but so far all the experiments and the early re-write are going well and it’s solving the issues I was having with Electron. This short post is about CircleCI.

Because now I’m using common, boring technologies, I decided setting a CI was going to be easy enough that I didn’t mind doing it and I chose CircleCI, mostly because of their free plan.

Sadly, my project didn’t work out of the box. JavaFX was nowhere to be found. The reason was that CircleCI uses OpenJDK instead of the Oracle’s JDK by default. I guess that’s enough for the vast majority of people. Switching to Oracle’s JDK was not hard, but I couldn’t find a straightforward post, so, this is it.

CircleCI gives you a configuration example that contains something like this:

version: 2
  jobs:
    build:
      docker:
        # specify the version you desire here
        - image: circleci/openjdk:8-jdk

I’m not very familiar with Docker and how CircleCI uses, all I know is that it’s a container system for, essentially, operating system images. CircleCI has a repo of Docker images and the one that include’s Oracle’s JDK is called circleci/jdk8. What comes after the colon in the config is the version of the Docker image (I think) which Docker calls tags and for this one, at the time of this writing, there was 0.1.1 but you can easily check the latest one here: https://hub.docker.com/r/circleci/jdk8/tags/

With that in mind, the config should now look like this:

version: 2
  jobs:
    build:
      docker:
        # specify the version you desire here
        - image: circleci/jdk8:0.1.1

And that’s it… well, unless you are using Gradle. That image doesn’t provide Gradle, so, you need to replace all the mentions of gradle with gradlew and use the wrapper that should be part of your repo. But before that will work, you need to make that file executable on the CircleCI server by calling chmod +x gradlew on it. The resulting config file for me looks like this:

# Java Gradle CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-java/ for more details
#
version: 2
jobs:
  build:
    docker:
      # specify the version you desire here
      - image: circleci/jdk8:0.1.1

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/
      # - image: circleci/postgres:9.4

    working_directory: ~/repo

    environment:
      # Customize the JVM maximum heap limit
      JVM_OPTS: -Xmx3200m
      TERM: dumb

    steps:
      - checkout
      - run: chmod +x gradlew

      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "build.gradle" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-

      - run: ./gradlew dependencies

      - save_cache:
          paths:
            - ~/.m2
          key: v1-dependencies-{{ checksum "build.gradle" }}

      # run tests!
      - run: ./gradlew test

And that’s it, that worked.

Setting (database) credentials on a Spring Boot project, the right way

Searching online for how to set up the credentials to access the database (or any other service) while in development leads to a lot of articles that propose something that works, but it’s wrong: putting your credentials in the application.properties file that you then commit to the repository.

The source code repository should not have any credentials, ever:

  • You should be able to make your project open source without your security being compromised.
  • You should be able to add another developer to your team without them knowing any credentials to your own development machine.
  • You should be able to hire a company that does a security analysis of your application, give them access to your source code and they shouldn’t gain access to your database.
  • You should be able to use a continuous integration service offered by a third party without that third party learning your database credentials.

If you want to see what happens when you commit your credentials to your repo, check out these news articles:

That’s probably enough. I hope I convinced you.

In an effort to find a solution for this, I asked in Stack Overflow and I got pointed in the right direction.

Leave application.properties where it is, in your resources of code folder, commit it to the repository. Instead, create a new file in ${PROJECT_ROOT}/config/application.properties and also add it to your version control ignore file (.gitignore, .hgignore, etc). That file will contain the credentials and other sensitive data:

# This should be used only for credentials and other local-only config.
spring.datasource.url = jdbc:postgresql://localhost/database
spring.datasource.username = username
spring.datasource.password = password

Then, to help onboard new developers on your project (or yourself in a new computer), add a template for that file, next to it. Something like ${PROJECT_ROOT}/config/application.template.properties that will contain:

# TODO: copy this to application.properties and set the credentials for your database.
# This should be used only for credentials and other local-only config.
spring.datasource.url = jdbc:postgresql://localhost/database
spring.datasource.username = 
spring.datasource.password = 

And voila! No credentials on the repo  but enough information to set them up quickly.

Disclaimer: I’m new to Spring Boot, I only started working with it a few days ago, so, I may be missing something big here. If I learn something new that invalidates this post, I’ll update it accordingly. One thing I’m not entirely sure about is how customary it would be to have ${PROJECT_ROOT}/config/application.properties on the ignore list. Please, leave a comment with any opinions or commentary.

My take on an AJAX re-frame effect handler

There’s already an AJAX re-frame effect handler provided by Day8, the same guy who made re-frame and there’s nothing wrong with it. From the documentation, this is how you use it:

(re-frame/reg-event-fx ::http-post
  (fn [_world [_ val]]
    {:http-xhrio {:method          :post
                  :uri             "https://httpbin.org/post"
                  :params          data
                  :format          (ajax/json-request-format)
                  :response-format (ajax/json-response-format {:keywords? true})
                  :on-success      [::good-post-result]
                  :on-failure      [::bad-post-result]}}))

That felt a little to verbose for my taste, so I made my own that you use like this:


(re-frame/reg-event-fx ::http-post
  (fn [_world [_ val]]
    {:ajax {:post            "https://httpbin.org/post"
            :params          data
            :on-success      [::good-post-result]
            :on-failure      [::bad-post-result]}}))

If you are familiar with cljs-ajax, you’ll notice the pattern. Day8’s solution uses the raw cljs-ajax API, while I use the higher level one. Day8’s is more versatile, mine is more compact.

This is my solution:

(ns ajax-fx-handler
  (:require [ajax.core :as ajax]
            [re-frame.core :as re-frame]
            [clojure.set :as set]))

(re-frame/reg-fx :ajax
  (fn http-effect [request]
    (let [request-verb (set/intersection #{:get :post :put :patch :delete :head :options :trace :purge}
                                         (set (keys request)))]
      (if (not= 1 (count request-verb))
        (throw (js/Error (str "When specifying an AJAX request, one and only one verb should be specified. Found: " request-verb)))
        (let [request-verb (first request-verb)
              url (get request request-verb)
              request (if (contains? request :on-success)
                        (assoc request :handler #(re-frame/dispatch (conj (:on-success request) %)))
                        request)
              request (if (contains? request :on-failure)
                        (assoc request :error-handler #(re-frame/dispatch (conj (:on-failure request) %)))
                        request)
              request (-> request
                          (dissoc request-verb)
                          (dissoc :on-success)
                          (dissoc :on-failure))
              ajax-fn (cond
                        (= request-verb :get) ajax/GET
                        (= request-verb :post) ajax/POST
                        (= request-verb :put) ajax/PUT
                        (= request-verb :patch) ajax/PATCH
                        (= request-verb :delete) ajax/DELETE
                        (= request-verb :head) ajax/HEAD
                        (= request-verb :options) ajax/OPTIONS
                        (= request-verb :trace) ajax/TRACE
                        (= request-verb :purge) ajax/PURGE)]
          (ajax-fn url request))))))

Feel free to use it, but, is it worth releasing it as a library? does anybody want that?

Pre-signed S3 URLs with Clojure and ClojureScript

Dashman uses S3 for storing the encrypted screenshots. They can go directly from the renderer application to S3 and from there to the displayer application as there’s no need for the server application to ever see them. They are end-to-end encrypted anyway.

My needs in this regard were a bit unique. I have a server component that can generate whatever tokens and signatures are needed and then send it to both applications. The applications are written in ClojureScript, so, essentially, JavaScript, but unlike most direct-to-S3 upload solutions, I don’t have a web site with a form, so, I don’t need to fiddle with CORS or forms.

The two libraries

I found two libraries that could have helped me. The big one is amazonica, a Clojure library to interact with AWS. It looks very complete but when I tried to use the com.amazonaws.services.s3/generate-presigned-url function, I got this error:

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

There are two versions of signatures, version 2 and 4. Version 2 is deprecated, it might or might not work on some data centers but it definitely doesn’t work on all. I refused to believe it at first because I was using very recent version of all libraries, but yup, it was generating version 2 of the signature with signed URLs that look something like this:

https://bucket-name.s3.amazonaws.com/key-name?AWSAccessKeyId=AKI...OA&Expires=1494190327&Signature=ct...3D

Digging further I found this snippet of code:

(System/setProperty SDKGlobalConfiguration/ENFORCE_S3_SIGV4_SYSTEM_PROPERTY "true")

But for me, it made no difference in my case. For the record I reported the issue and it is supposed to work, but it doesn’t for me.

Then there’s s3-beam, that looked promising, but it seems targeted towards forms, not just plain PUT requests from the clients. On top of that, I didn’t want to mess with async, I didn’t need it and it would be over-engineering for my case. Lastly, it uses it’s own custom way of creating the signatures instead of Amazon’s SDK and that code looks like it’s generating the old signatures (sha1 instead of sha256). I ended up not even trying it.

Instead, I decided to do my own custom option, which was not hard and worked straight away. This post is about sharing that code.

Custom solution using the AWS Java SDK

Starting on the server/Clojure side of things, I added the dependency:

[com.amazonaws/aws-java-sdk "1.11.126"]

With only that I was getting this unexpected error when trying to use the AWS SDK:

ERROR com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
ERROR Could not initialize class com.amazonaws.partitions.PartitionsLoader
which I resolved by adding these other dependencies as well (mostly about getting the latest version):
[com.fasterxml.jackson.core/jackson-core "2.8.8"]
[com.fasterxml.jackson.core/jackson-databind "2.8.8"]
[com.fasterxml.jackson.core/jackson-annotations "2.8.8"]

I wrote a function to generate the S3 client:

(defn aws-s3-client []
  (let [region (Regions/fromName (config/env :aws-s3-region))
        credentials (BasicAWSCredentials. (config/env :aws-access-key-id)
                                          (config/env :aws-secret-access-key))]
    (-> (AmazonS3ClientBuilder/standard)
        (.withRegion region)
        (.withCredentials (AWSStaticCredentialsProvider. credentials))
        .build)))

config/env just fetches configuration values, powered by cprop and mount. This is just standard stuff from Luminus. With

(defn- pre-signed-url [key & [verb]]
  (let [expiration (tc/to-date (-> 24 t/hours t/from-now))
        generate-presigned-url-request (doto (GeneratePresignedUrlRequest. (config/env :aws-s3-bucket-name) key)
                                         (.setMethod (if (= verb :put)
                                                       HttpMethod/PUT
                                                       HttpMethod/GET))
                                         (.setExpiration expiration))]
    (.toString (.generatePresignedUrl (aws-s3-client) generate-presigned-url-request))))

The function generates either a URL for reading, :get, or for writing, :put, depending on the parameter verb. key is just the path part of the URL (remember, S3 is actually a key/value storage). All URLs will last for 24hs, something you can tweak as well.

And that’s it, generating the URL is done, now we have to use it.

Reading and writing from ClojureScript

If you have a form, then, you might need to deal with CORS and other issues. I suggest you look at s3-beam at least for inspiration. For me, it was just a matter of generating the requests from the ClojureScript app, which uses cljs-ajax. Reading was straightforward enough:

(ajax/GET get-url
          {:handler (fn [value]
                      (do-something value))})

although the handler might have to deal with encoding issues. Writing is similar, but again, you may need to fiddle with encoding issues. Since what I was sending is a JSON payload (an encrypted screenshot), this worked for me:

(ajax/PUT put-url 
          {:format :json
           :params payload})

If you are uploading binary data, I recommend you play with :format :raw and the :body attribute.