Pablo Fernandez

One of my projects, Unbreach, has a database of more than 600 breaches. These come from haveibeenpwned and they are composed of some metadata, a one-paragraph description, and an image. I wanted to improve these with more content, links to articles, tweets, videos, and some content of my own.

I decided that a good way to do it would be to move them from the app, which resides at app.unbrea.ch, to the marketing website, which is at unbrea.ch, essentially creating them as blog posts. That way after the blog post is automatically created (when haveibeenpwned ads the breach), I can go in and manually edit it in all the WordPress glory. I thought this was going to take me a few hours, not days.

Hopefully, with this blog post, it’ll only take you hours. I’ll be using Ruby but it should be trivial to translate it to Python, JavaScript, or any other programming language. Writing the code wasn’t the hard part, understanding the WordPress.com world was.

WordPress has two different APIs that should be able to accomplish this task, one is the XML-RPC API and the other is the REST API. The XML-RCP API depends on a file called xmlrpc.php and it’s strongly recommended you leave this deactivated because it had a lot of security issues. It’s also old, cumbersome, and possibly on the way out. I didn’t want to use it and I don’t think you should either.

From what I can gather the REST API is what the admin tool uses, so using it sounds like a safe bet. If you are going to be creating blog posts from an unattended background process, as I do, you’ll find your first obstacle when you read about authentication because it just assumes there’s a browser sending cookies.

Fear not! There are plug-ins that implement other authentication methods and one of those is the Application Passwords plug-in. Which is now discontinued because it’s been merged into WordPress itself in version 5.6. This sounds promising until you realize the feature seems to be missing in WordPress.com.

If you search how to create an Application Password on WordPress.com you’ll land in the wrong place. WordPress.com users have an Application Password that’s hidden behind the Two-Step Authentication in Security. This is what it looks like:

What’s going on here? Well, WordPress.com has its own API, which is a REST API, and if you talk to support and WordPress.com they’ll point you to that. I wasn’t a fan of that solution because although I want to use WordPress.com, I don’t want to be tied to it. I want to be able to move to WP Engine or something like that whenever I want.

That API, similar to the REST API, assumes there’s a human interacting through a third-party application, so it’s not great for unattended processes. Authentication works using OAuth2 which for a background job that just needs an API key I find very annoying. It’s doable but annoying. Well… it’s doable until you enable 2FA and then it’s not doable anymore, and that’s why that specific Application Password exists.

WordPress.com support also told me that the WordPress REST API is enabled only if you are on a business plan or above.

So… where’s the Application Password for the REST API then? I don’t know if there’s a link to it anywhere, but you get to it by going to https://example.com/wp-admin/profile.php where example.com is the URL of your blog. That is, add /wp-admin/profile.php to it. On WordPress.com’s defense, it was their support that finally pointed me to it. When you go there you’ll see an old-style profile page:

The previous Application Password was tied to the user, this one is tied to the user and the site, so if you have more than one site you’ll need to create one per site.

And that was the hard part. Once I got that application password things just worked. It’s a straightforward and mostly well-documented API. I’ll share my messy code here anyway (sorry, didn’t have time to clean it up).

In Ruby I’m using a library called Faraday to talk to APIs. The first thing is creating the Farady object that has the metadata that will be used in all the requests:

auth_token = "#{Rails.application.credentials.wordpress&.username}:#{Rails.application.credentials.wordpress&.app_pass}"
auth_token = Base64.strict_encode64(auth_token)
conn = Faraday.new(url: ENV["WORDPRESS_URL"],
  headers: { "Authorization" => "Basic #{auth_token}" }) do |conn|
 conn.request :json
 conn.response :json
end

According to Faraday’s documentation, this should have worked as a better way of setting up the authentication details:

conn.request :authorization,
             :basic,
             Rails.application.credentials.wordpress&.username,
             Rails.application.credentials.wordpress&.app_pass

but for me it didn’t. It was completely ignored. About those two values, Rails.application.credentials.wordpress&.username is the username of the user that will be creating the posts and Rails.application.credentials.wordpress&.app_pass is the corresponding application password. ENV["WORDPRESS_URL"] is the URL of the WordPress site, like https://unbrea.ch/.

The first thing I need is the id of the category in which these posts will end up. This is very important because they appear on a separate page about breaches and not on the blog and that’s achieved with categories:

response = conn.get("/wp-json/wp/v2/categories", {search: "Breach", _fields: %w[id name]})
if response.status != 200
  raise "Unexpected response #{response.status}: #{response.body}"
end
category = response.body.find { |category| category["name"] == "Breach" }

Now, if the category doesn’t exist, I want to create it:

if category.nil?
  response = conn.post("/wp-json/wp/v2/categories") do |req|
    req.body = {name: "Breach"}
  end
  if response.status != 201
    raise "Unexpected response #{response.status}: #{response.body}"
  end
  category = response.body
end

Then I needed to do the same with tags. In my case, the tags were in a field called data_classes and the code for getting the id of the tag and creating it if it doesn’t exist is very similar:

tags = data_classes.map do |data_class|
  response = conn.get("/wp-json/wp/v2/tags", {search: data_class, _fields: %w[id name]})
  if response.status != 200
    raise "Unexpected response #{response.status}: #{response.body}"
  end
  tag = response.body.find { |tag| tag["name"] == data_class }

  if tag.nil?
    response = conn.post("/wp-json/wp/v2/tags") do |req|
      req.body = {name: data_class}
    end
    if response.status != 201
      raise "Unexpected response #{response.status}: #{response.body}"
    end
    tag = response.body
  end

  tag
end

And finally, we can create the post. I create the content as an HTML snippet which causes WordPress to interpret it as classic content, not as blocks. But that’s fine because it renders well and the first time I edit one of those posts converting them to blocks is two clicks and works perfectly for this simple content.

content = <<~CONTENT
  <p>#{description}</p>
  <p><!--more--></p>
  <p>Accounts breached: #{pwn_count}</p>
  <p>Breached on: #{breach_date&.strftime("%B %d, %Y")}
  <p>Exposed data: #{data_classes.to_sentence}</p>
  <p>Domain: #{domain}</p>
  <p>Added on: #{added_date.strftime("%B %d, %Y")}</p>
CONTENT

response = conn.post("/wp-json/wp/v2/posts", {
  title: title,
  content: content,
  excerpt: description,
  status: "publish",
  categories: [category["id"]],
  tags: tags.map { |tag| tag["id"] },
  date_gmt: (breach_date.to_time(:utc) + 12.hours).iso8601.to_s,
  template: "breach-template",
  ping_status: "closed"
})
if response.status != 201
  raise "Unexpected response #{response.status}: #{response.body}"
end
post = response.body

At this point, I wasn’t done. I wanted these posts to have the image associated with the breach (the logo of the company breached). The first step was downloading it which was a trivial one-liner:

logo_request = Faraday.new(url: logo_path).get("")

In that code, logo_path is actually a full URL of the file.

To create media items in WordPress, I needed to encode the post as multi-part, so I ended up creating a separate Faraday object for that:

multipart_conn = Faraday.new(url: ENV["WORDPRESS_URL"],
  headers: {"Authorization" => "Basic #{auth_token}"}) do |conn|
  conn.request :multipart
  conn.response :json
end

It should have been possible to use a single Faraday object for all requests, but when you specify multipart, you need to take care of encoding the JSON requests yourself and adding them as one of the parts. This is where I got lazy and just moved on with my work.

The code for creating the image in WordPress is this:

extension = File.extname(logo_path)
file_name = "#{name.underscore.tr("_", "-")}#{extension}"
content_type = if extension == ".png"
  "image/png"
else
  raise "Unexpected extension #{extension}"
end
media = multipart_conn.post("/wp-json/wp/v2/media", {
  date_gmt: (breach_date.to_time(:utc) + 12.hours).iso8601.to_s,
  status: "publish",
  title: title,
  comment_status: "closed",
  ping_status: "closed",
  alt_text: "Logo for #{title}",
  caption: "Logo for #{title}",
  description: "Logo for #{title}",
  post: post["id"],
  file: Faraday::Multipart::FilePart.new(StringIO.new(logo_request.body), content_type, file_name)
})

In reality, 100% of the images are PNG so I was ok with such a simplistic approach. When creating the FilePart I wrapped logo_request.body in a StringIO because it already contained the binary data of the image. If you have a local file you can just pass the path to FilePart.new and it just works.

And now that I had the image, I could set it as the featured image for the post I created earlier:

response = conn.post("/wp-json/wp/v2/posts/#{post["id"]}", {
  featured_media: media.body["id"]
})
if response.status != 200
  raise "Unexpected response #{response.status}: #{response.body}"
end

The reason why I didn’t create the image before creating the post was so that I could pass the post id to the image and thus the image would be connected to the post. I’m not sure how useful that is.

And that’s all.

I wonder if this code should be put in a gem and made reusable. WordPress points to the wp-api-client gem as the Ruby solution, which is read-only and abandoned. There’s also wordpress_v2_api, but I wasn’t a fan of the API (it’s almost like using HTTP directly), it hasn’t been touched in 6 years and I don’t believe it supports writing. I’m half tempted to fork wp-api-client, but does anybody else care, or is it just me? Please leave a comment if this is something you want to use.