AJ Christensen

Senior Site Reliability Engineer,  brightwheel

March 2019 —  December 2019   10 months

• Capacity planning and improvements for 'back to school' busy period
• Significant security hardening of monolithic Rails application, continuous validation
• Maintain and improve automated Incident Response Slack bot and processes
• Establish brightwheel API Service Level Objectives (SLOs), improving feedback loop from user behavior
• Evaluation of distributed tracing technologies: NR Dist Trace, Scout, Honeycomb and LightStep, implementation of OpenTracing/Telemetry
• Evaluation of Service Mesh and Canary Release techniques and technologies: linkerd and flagger
• Improvements to application and infrastructure continuous integration and Docker image delivery pipelines; automated security scanning

Principal Site Reliability Engineer,  DoubleDutch, Inc.

July 2017 —  February 2019   2 years

Remote Operations Engineer contract was requisitioned for various systems engineering duties and on-call emergency response, starting with developing infrastructure automation for the existing systems and deployment tooling in various stages of maintenance.

The DoubleDutch systems have web-facing front-end CMS and API monolithic C#.NET components, large physical PostgreSQL databases, full text indexing, and queuing systems, as well as roughly one hundred other features/components running in containers, with white-labeled mobile applications for both Android and iOS.

With baseline automation repaired or re-established, my team was allocated to an experimental platform restructuring project which was being built by a remote team in Amsterdam with Java and Docker (compose).

While that project was ultimately canceled, we produced the systems automation for the virtualized infrastructure in Rackspace's OpenStack public cloud with Chef and Docker, and at the same time ported the Docker image builds to Google Cloud Build, and wrote Helm charts for every component to run in Kubernetes at availability and responsiveness levels measurably exceeding any previous capability, with significantly reduced human intervention. Much of this would be later re-usable.

Later, a new Platform team was formed, absorbing the previous Engineering Operations, Integrations, and Data teams. Our function was to migrate the legacy infrastructure components while building any new components in our newly designed Google Cloud Platform / Google Kubernetes Engine infrastructure and tooling.

• Design, deployment and maintenance of all existing physical and virtual Infrastructure with Chef.
• Design of, and iterative migration towards Google Kubernetes/Compute based infrastructure in Iowa, Belgium and Frankfurt.
• Designed and implemented redundant dedicated interconnect hybrid connectivity solution between Rackspace RackConnectGlobal Equinix and Google Compute Platform.
• Designed and implemented redundant VPN/IPSec hybrid connectivity solution between Rackspace regional ServiceNets and Google Compute Platform.
• Migrated in-house Identity authentication and authorization system and associated database clusters and workloads to Google Cloud Platform and Kubernetes.
• Development of Helm charts repository for container (Docker) workloads.
• Extensive usage of Google Cloud Build for all container workloads including continuous integration of pull requests from GitHub Enterprise with assets stored in the Google Container Registry for continuous vulnerability and exploit scanning.
• Database administration of PostgreSQL clusters with pooling, replication management and backup/continuous archiving.
• Maintenance/support of legacy software systems and application deployment tooling.

Site Reliability Engineer,  Pantheon Systems, Inc.

July 2014 —  June 2016   2 years

Platform Infrastructure 'Platfrastructure' contractor providing reliability engineering services for over 200,000 individual Git version-controlled websites, primarily Wordpress and Drupal, custom Python/Cassandra based configuration/customer management system, powered at the infrastructure level by our custom edge router Styx (comprising Varnish and Go), mutual TLS capabilities, and Chef-based systemd cgroup/namespace isolation of NGINX, PHP and MySQL all running on the Rackspace OpenStack public cloud virtual CentOS hosts.

• Development through to completion, and deployment of, proprietary Go / Varnish HTTP edge router: 'Styx'.
• Worked with team to improve Infrastructure as Code concepts: migration from Chef cookbook monolith to individual Chef cookbook continuous delivery pipelines with Berkshelf and Baryon
• Maintenance, design and development of on-call/alerting/monitoring systems - Sensu, Graphite+Grafana, ELK et al. Brief exploration of InfluxDB.

Operations Engineer,  MoPub | Twitter

July 2013 —  July 2014   a year

• Design/maintenance of Chef systems automation for MoPub software and services
• Expansion into Chef/Opscode Omnibus technology for Application service delivery; relevant cookbook re-design
• Training other Operations Eng. on advanced Chef software automation practices, Chef upgrades, LWRPs/HWRP/Libraries.

Engineer,  Heavy Water Operations

June 2011 —  June 2013   2 years

After occasional moonlighting with Darrin Eden's Vancouver-based consulting firm Heavy Water Software and his relocation to Oregon, we decided to join forces and expand the team considerably, bringing development operations and software development consulting services to numerous clients, while unifying our service and product offerings and methodologies.

• Client engagements of all shapes and sizes, skill sets and experience, geographic distribution
• High velocity front-line consulting in development/operations/engineering roles
• Open source project maintenance -- Heavy Water Chef cookbooks
• Experimental product development -- Container systems, Heavy Water "Fission" and "Pennyworth" continuous build systems.
• Internal marketing efforts -- site, product, services

Rotorua Drone Racers (inactive) , Race Director

May 2018 —  Present   2 years

Race Director for Rotorua Drone Racers Winter Series monthly events.

• Built Delta5 RF-based timing system using LiveTimeFPV
• Build and maintenance of drone racing obstacles: gates, flags, etc.
• Ran multiple events over the winter season while competing at the same time.

Betaflight , Maintainer

December 2017 —  Present   2 years

Part of the Betaflight Member group. Involved in both embedded software development on STM32 micro-controllers, ARM SDK build system tooling, and of course a lot of flight testing and support, it's the perfect combination of hobbies.

• Market leader in hobbyist grade flight controls software (firmware).
• 83,000+ Monthly unique users of Configuration software.

Drone Racing New Zealand (previously Rotorcross N.Z.) , Chapter Manager (East Coast and Lakes)

January 2014 —  January 2016   2 years

Coordination and support of First Person View drone racing events across New Zealand.

• East Coast and Lakes chapter reached 'Tier 2' MultiGP sponsorship status which requires running a certain number of events with continuous minimum participation.

Chef, Inc. (Opscode) , Software Engineer

March 2008 —  Present   12 years

Independent contributor / maintainer of numerous original Chef cookbooks and extension library code.

• Second contributor to the Chef code-base having met with founder Adam Jacob working together on the Puppet project.
• Numerous open-source community cookbooks and adoption

0.5.4, 0.6.0 MVP, Opscode, Inc.

Awarded on: Apr 29, 2009

Release MVP for outstanding contributions to the Chef project.