bobbyratliff

Creating structures that automate things people do every day.

Archive for March, 2026

jq Cheat Sheet

Convert JSON to CSV using jq

Input JSON (products.json)

{
  "products": [
    {
      "sku":"Carrot",
      "description":"bunches of 6 to 8. Pack in 1 1/9 bu. box"
    },
    {
      "sku":"Broccoli",
      "description":"1 1/9 bu. box. Weight 23 lb. Stem on. Iced."
    }
  ]
}

Convert using jq:

jq -r '.products | .[] | [.sku, .description] | @csv' < products.json

Limitations: No CSV header

Convert using yq

yq is a command-line YAML, JSON processor with handy default behavior. Install using homebrew:

brew install yq

Convert JSON to CSV using jq

% yq -o=csv products.json 
Error: csv encoding only works for arrays, got: !!map

First attempt failed. Looks like I need to give it an expression to select “products”

% yq -o=csv '.products' products.json
sku,description
Carrot,bunches of 6 to 8. Pack in 1 1/9 bu. box
Broccoli,1 1/9 bu. box. Weight 23 lb. Stem on. Iced.

Success! More yq documentation on their website.

How we made monitoring simple with TIG stack

TIG is a popular stack to use for monitoring, including Telegraf, InfluxDB, and Grafana.

  • Telegraf is a monitoring agent that can collect metrics and send them to a destination.
  • InfluxDB is a time series database, optimized for storing data points associated with a timestamp, and querying those data points based on a time range.
  • Grafana is a dashboard and visualization tool.

When we started our monitoring project, we inherited a TIG stack with a separate influxdb instance per region. It did 90% of what we needed, but had a complicated deployment strategy and was not easy to modify. The goal was to consolidate to a single InfluxDB instance per tier (QA, Staging, Prod) for cost savings and aggregated reporting.

During this project, the Telegraf config didn’t change very much. It was pretty simple: an Influx line protocol input, and an Influx output using authentication. The Telegraf config file was managed using Salt. We would change the output from the per-region influxdb instances to the shared instance.

The shared InfluxDB instance would be in a US-based region which would result in higher latency from international services. This was an acceptable trade-off and I believed that telegraf would be able to handle the additional latency without causing issues for the applications we monitored.

First step: Deploy the new InfluxDB instance

  • This was the simplest step: add the new instance as a new “region” alongside the existing regions.
  • We stored the credentials in the same secrets manager as the original instances.

Second step: Update Telegraf config to point to new InfluxDB instance

  • Most telegraf configs were deployed with Salt, and a few were deployed with Ansible inside an AMI. Either way, updating the config was straightforward.
  • While updating Telegraf config: Add tagging to indicate the origin of the telemetry. We called this the dc tag because it indicated which datacenter the metrics were coming from. This environment tag preserved the previous ability to report on an individual environment.

Third step: Iterate and make sure all instances are cut over to the new shared instance

  • I wrote an InfluxQL query to check for the existence of telemetry with a recent timestamp. This was the canary that could tell if a regions-specific instance was still being used.

Fourth step: Destroy the region-specific instances

  • Save money, make things simpler.
  • A multi-tenant configuration.

Gotchas:

  • During the migration, we also decided to refactor some Ansible, moving one of the roles from a shared base module into the specific playbooks that needed it. This resulted in a gotcha, where that AMI was using apt-get to install the latest version of telegraf rather than a pinned version that the shared base module had been installing. To resolve this, we went into the telegraf config and updated some config items from deprecated names to the new names used by the latest version of telegraf.

Pandoc

Convert Markdown to Slides

  1. Write Markdown, using headings to chunk your file into sections. Each heading will be its own slide.
  2. Annotate the markdown file with YAML by adding a document header:
title:
 - RAG Pipeline
author:
 - Bobby Ratliff
theme: 
 - Copenhagen
date:
 - March 4, 2026

Then convert the markdown file to html:

pandoc -t slidy -s rag-pipeline.txt -o rag-pipeline.html

For more information, see the Pandoc Manual.

Convert Markdown to PDF

Install dependencies

brew install pandoc
brew install --cask basictex
# Ensure latex utilities are on the PATH
eval "$(/usr/libexec/path_helper)"

Convert the document

pandoc -s 'rag pipeline.md' -o 'rag pipeline.pdf'

Spring Boot Cheat Sheet

Spring Cloud Config

Web Debug

Example: Log the reason a 400 bad request is being produced. This will log which ControllerAdvice method is being called.

logging.level.org.springframework.web.servlet.mvc.method.annotation=DEBUG

Database Debug

spring.jpa.show-sql=true
logging.level.org.hibernate.SQL=DEBUG
logging.level.org.hibernate.type.descriptor.sql.BasicBinder=TRACE
logging.level.org.springframework.transaction=TRACE
logging.level.org.springframework.jdbc.core=TRACE

Security Debug

logging.level.org.springframework.security=DEBUG
logging.level.org.springframework.web=DEBUG

What I Learned Integrating OAuth 2.0 Social Login with Spring Boot

Spring Security does a lot of things out of the box.

  • Most of OAuth 2.0 Login can be turned on with a handful of configuration properties.

Third party OAuth 2.0 providers make it easy to integrate

  • They provide a way to configure the clientID and client secret needed by Spring Security.
  • You can store this information in GitHub Actions Secrets so that it’s not checked into your codebase. (See bike-tracker repo for an example)

Keep tenancy simple

  • Part of the project involved an internal custom OAuth Provider, which I was responsible for integrating with another internal product. A mistake I made at first was to require a per-tenant configuration. From a security point of view, this was a very secure model because it enforced tenant isolation. A provider OAuth integration would be created inside the customer’s tenant, and then the downstream
  • The per-tenant configuration could be paired with a username lookup to ensure the user was in the same tenant. However, it was tedious to configure because the OAuth integration had to be configured inside the customer tenant and had to match the OAuth configuration in the other product. In the future I would have a shared OAuth configuration that could be used for multiple tenants.