JMD
Dashboard Platform

Dashboard Platform


My graduation internship, February to August 2024, at a large international company.

Problem

A bunch of teams were each running their own Grafana in their own AWS account, on different versions, mostly idle, each with its own login. The company wanted them merged into one, mainly for cost and security.

Before: each team with its own Grafana setup

Solution

One shared Grafana on AWS where every team gets its own organization and manages its own users. It went to production and is still used.

Choosing Grafana

I compared Grafana with Metabase, Apache Superset and Redash. All of them can keep teams apart, but Grafana was the only one the existing dashboards could be moved into, and every team already knew it.

Infrastructure

Everything is in Terraform, split into modules for the VPC, security groups, IAM, load balancer, ECS, EC2 and RDS. Grafana runs on ECS on an EC2 Auto Scaling group across two availability zones, with its data in RDS PostgreSQL. Deploys run through CodePipeline and CodeBuild.

Architecture

I picked EC2 over Fargate because of cost, and moved Grafana’s data out of SQLite into Postgres so containers can be replaced without losing anything. Before settling on a scaling policy I load tested a few, based on CPU and memory.

Platform components

Login

Cognito is connected to the company’s Azure AD, so people log in with their normal account. The load balancer blocks anyone who isn’t signed in, and a second OAuth client logs them into Grafana, because team owners had to be able to manage roles inside Grafana themselves. Only specific Azure AD groups get past the load balancer.

Login flow

Rollout

After the MVP I ran product reviews with several teams, demoed it to the department and had a senior engineer review the Terraform. I also wrote the IAM policies the platform needed, which were rolled out to all of the company’s AWS accounts, plus user tutorials and a migration plan for the old Grafana instances. The last part was a production readiness checklist based on the AWS Well-Architected Framework, with automated system tests for login, database connections and dashboard loading.