A serverless, data-driven Git metrics dashboard using Amazon Quick Sight

extract

Artificial Intelligence A serverless, data-driven Git metrics dashboard using Amazon Quick Sight Git activity is one of the richest signals engineering teams produce that can provide continuous observability into development analytics. The challenge is extracting these Git metrics at scale, which has traditionally required hand-rolled extract, transform, and load (ETL) jobs, dedicated infrastructure, and ongoing maintenance. Further, with modern development tools becoming more prevalent, teams need a clear way to measure whether these tools are making developers faster, or if the investment is not paying off. The AWS AI-Driven Development Lifecycle (AI-DLC) framework makes a simple point: If you’re using AI coding tools, you need numbers to back it up. Set a baseline, track what changes, and keep watching. Otherwise, you’re flying blind. You won’t know whether AI is speeding things up, only padding the commit counts, or quietly introducing quality issues you did not expect. In this post, we introduce a serverless approach that automates Git metrics collection from GitHub and GitLab to surface live analytics using Amazon Quick Sight. Whether you’re tracking sprint velocity, assessing release readiness, or you need visibility into team patterns, this solution provides near-real-time insights into your Git platform activities. The serverless design abstracts infrastructure management while staying low-cost at scale. Solution overview This solution implements an automated, event-driven pipeline that collects repository metrics from GitHub and GitLab APIs automatically on schedule. It then processes them through a serverless orchestration workflow and persists the results in Amazon Simple Storage Service (Amazon S3) for graphical visualization using interactive Amazon Quick Sight dashboards. The key capabilities include: - Intelligent change detection – A dedicated detector function monitors GitHub events API and GitLab activity feeds to determine whether changes like commits, pull requests, issues, repository creation or deletion occurred since the last collection. If no changes are detected, the processing is skipped entirely. - Adaptive chunking – For organizations managing more than 20 repositories, the solution automatically partitions the workload into parallel chunks and processes them concurrently using AWS Step Functions Map states. - Full and incremental loads – On first execution, the system performs a full load of all repository metadata. Subsequent runs use incremental logic, collecting only data that has changed. A full refresh is automatically triggered every 24 hours to maintain data accuracy. - Configurable scheduling – How often metrics get collected is up to you. It’s an AWS CloudFormation parameter that accepts both rate() andcron() expressions, so you can set whatever schedule fits. The solution aligns with the AI-DLC framework’s observability pillar by providing continuous, automated measurement of development activity. You can establish velocity baselines before adopting AI tools, track changes during rollout, and quantify improvements over time. Architecture The architecture uses six core AWS services to build a fully managed, event-driven pipeline: - Amazon EventBridge Scheduler: A scheduled rule starts the workflow at the interval you configured. It runs the collection cycle automatically without any external triggers or manual intervention. - AWS Step Functions: A Step Functions state machine orchestrates the entire collection workflow. It first invokes the change detector to determine whether new activity has occurred. Based on the result, it decides between a full load and an incremental load. A full load collects all repository metadata from scratch on the first run and every 24 hours thereafter. An incremental load collects only data that changed since the last run, which reduces API calls and execution time. The workflow then evaluates the number of active repositories. If the count exceeds the chunking threshold (default: 20), it partitions the repositories into equal-sized chunks and processes them concurrently using a Step Functions Map state. Each chunk is handled by a separate AWS Lambda invocation. If the count is below the threshold, a single direct invocation collects all repositories. The workflow includes comprehensive retry logic with exponential backoff for transient API failures. - AWS Lambda: - Change detector – github-change-detector : This Lambda function queries the activity feeds to determine the load type. It monitors six categories of events: push events, pull requests, issue creation and updates, repository creation, repository deletion, and contributor changes. If no relevant events are found since the last check, the workflow terminates early. - Metrics collector – github-metrics-collector : This Lambda function performs the actual data collection. It operates in three modes:collect_all for direct processing,collect_chunk for processing a subset of repositories during parallel execution, andaggregate for combining the chunk results. - Change detector – - AWS Secrets Manager: Git tokens are stored securely in Secrets Manager. Both Lambda functions retrieve tokens at runtime, so that credentials are never hard coded or passed as environment variables. - Amazon S3: Collected metrics are stored on Amazon S3 with versioning and server-side encryption enabled. The output includes a structured JSON file containing the full API response with nested repository details, and a flattened CSV file optimized for analytics. Both files include fields such as repository name, total commits, open and closed pull requests, open and closed issues, number of contributors, primary language, last activity timestamp, and repository creation date. S3 serves as the durable, cost-effective data store for all historical metrics. - Amazon Quick Sight: Reads the collected CSV data from the S3 data store or through Amazon Athena for SQL-based queries. It loads the data into SPICE (Super-fast, Parallel, In-memory Calculation Engine), a fast in-memory layer, to build interactive dashboards. You can build visualizations for summary metrics, pull request (PR) trends, development activity over time, per-repository drilldowns, and contributor analysis. The following figure provides an overview of the solution architecture using the listed services. Before moving on to the step-by-step walkthrough, let us look at the prerequisites required for you to follow along. Prerequisites - An active AWS account with permissions to create CloudFormation stacks, Lambda functions, S3 buckets, AWS Identity and Access Management (IAM) roles, Step Functions state machines, Amazon EventBridge rules, and Secrets Manager secrets. - AWS Command Line Interface (AWS CLI) v2. - A GitHub or GitLab account with repositories to monitor, and the ability to generate Personal Access Tokens. - An active Amazon Quick Sight subscription on either Standard or Enterprise edition for dashboard creation. - Least-privilege permissions for deploying the required resources: cloudformation:, s3:, lambda {CreateFunction,UpdateFunctionCode, UpdateFunctionConfiguration,GetFunction, PutFunctionConcurrency}, states:{CreateStateMachine,UpdateStateMachine,DescribeStateMachine}, events:{PutRule,PutTargets,DescribeRule}, sqs:{CreateQueue,GetQueueAttributes, SetQueueAttributes}, sns:{CreateTopic,GetTopicAttributes,SetTopicAttributes}, kms:{CreateKey, CreateAlias, PutKeyPolicy, EnableKeyRotation}, secretsmanager:{CreateSecret,GetSecretValue}, and iam:{CreateRole, PutRolePolicy, AttachRolePolicy, PassRole}. - Clone the solution from this GitHub repository. Navigate to the cloned directory, subsequent steps reference file paths relative to this root. Walkthrough In this section, we elaborate the steps that you can follow to deploy a sample dashboard in your test environment. Step 1: Create personal access tokens Before deploying the infrastructure, you need API tokens for each Git platform you plan to monitor. For GitHub: - Navigate to GitHub, Settings, Developer settings, Personal access tokens and choose Tokens (classic). - Choose Generate new token (classic). Under scopes, select only the minimum required permissions: repo (read-only access to repository metadata, commits, and pull requests) andread:org (read-only access to organization membership). Copy the generated token and store it securely. For GitLab: - Navigate to GitLab, Settings and Access Tokens. - Create a new token with scopes and copy the generated token. The following image shows the GitHub developer settings page where you can generate the new tokens: Step 2: Store tokens in AWS Secrets Manager Store each token as a secret in AWS Secrets Manager using the AWS CLI. See the following sample commands: Step 3: Deploy the CloudFormation stack Download this sample CloudFormation template to follow along and deploy the solution in your non-production environment. Option 1: Deploy using the AWS CLI. Replace placeholder values enclosed in angle brackets (<…>) with your actual AWS account ID, Region, and the secret ARNs from Step 2: Option 2: Deploy using the AWS Management Console. - Navigate to the AWS CloudFormation console. - Choose Create stack, With new resources (standard). - Upload the template, fill in parameters, and choose Submit. Stack reaches CREATE_COMPLETE status in 3-5 minutes. It creates the following resources: an S3 bucket with versioning and encryption enabled, two Lambda functions (Python 3.13 runtime), an IAM execution role with least-privilege permissions, a Step Functions state machine, an Amazon EventBridge schedule rule, and associated IAM roles. The following image shows the sample Create stack parameters and configuration: Step 4: Upload the Lambda deployment package The CloudFormation template deploys placeholder code for the Lambda functions. Update them with the actual logic using the pre-built deployment package from the cloned repository. The file lambda-package.zip is located in the deployment/ directory of the repository you cloned in the prerequisites. Replace <YOUR_BUCKET_NAME> in the following commands with the S3 bucket name created by your CloudFormation stack: Step 5: Execute and validate Trigger the workflow manually to verify end-to-end functionality. Replace <ACCOUNT_ID> with your 12-digit AWS account ID and <REGION> with your deployment AWS Region (for example, us-east-1): Monitor the execution in the AWS Step Functions console. A successful run produces a SUCCEEDED status within a few minutes. Verify that response.json file appears in your S3 bucket under the output/ prefix. Step 6: Connect Amazon Quick Sight to S3 data store to create your dashboards - Create a manifest file named manifest.json pointing to your S3 bucket and JSON file. See Supported formats for Amazon S3 manifest files for file format information. Replace<ACCOUNT_ID> and<REGION> with your values. Use the following sample manifest: - Go to the Amazon Quick Sight console. - Navigate to Data, Data sources, and choose Create data source. - Choose Amazon S3 and Next. - Provide a Data source name, upload your manifest file, and choose Connect. Note: If you receive a permission error stating that Amazon Quick Sight does not have access to the S3 bucket, you must first authorize Quick Sight to access your S3 resources. Navigate to Quick Sight, choose your account name, select Manage Quick Sight, then choose Security & permissions. Under Quick Sight access to AWS services, choose Manage, select Amazon S3, and add your metrics bucket. For detailed instructions, see Managing Amazon Quick Sight permissions to AWS resources. - Navigate to Data, Datasets, and then choose Create dataset. Choose your Data source name and Select. - The Data source details window opens. Choose Visualize. - This creates a new analysis under Analyses. Choose the Git-QS-Datasource analysis. - Create the interactive dashboard visualizations by changing the visual type and choosing fields such as total commits, pull requests, issues, PR merge trends, and active compared to stale repositories.Note: Sample code to create a dashboard is available in the accompanying GitHub repository. The following images show a sample dashboard with sample data around Repository Portfolio Analysis including total number of repositories, total commits, pull requests submitted, issues tracked, and so on. Step 7: Optionally configure alerts Set up Amazon CloudWatch alarms to monitor pipeline health. For example, create an alarm that triggers when Step Functions execution failures exceed zero in a one-hour window, sending a notification to an Amazon Simple Notification Service (Amazon SNS) topic for your operations team. Clean up To avoid ongoing charges, remove all resources when you no longer need the solution: Step 1: Empty and delete the S3 bucket. Step 2: Delete the CloudFormation stack. Step 3: Delete the secrets from Secrets Manager. Step 4: Remove Quick Sight datasets and Analyses (if created) from the Quick Sight console. After these steps, no resources from this solution remain in your account, and no further charges will accrue. Conclusion In this post, we demonstrated how to build a fully serverless Git analytics solution that automatically collects development activity data from GitHub and GitLab, persists it in Amazon S3, and visualizes it through interactive Amazon Quick Sight dashboards. The result is a lightweight, low-cost engineering intelligence solution with near-zero operational maintenance. Engineering managers gain sprint status visibility, product managers gain release confidence, and developers gain contribution awareness, all without managing any infrastructure. For teams adopting modern development agents, this solution also aligns with the AI-Driven Development Lifecycle (AI-DLC) methodology. It provides the continuous measurement layer needed to set baselines, track velocity shifts, and quantify real return on investment (ROI). Metrics such as commits per sprint, PR throughput, review cycle time, and contributor diversity help determine whether AI tools are driving meaningful delivery improvements. Getting started To deploy this solution in your AWS account, clone the sample starter kit repository and follow the provided instructions. To learn more about the components used in this solution, see the following resources: - AI-Driven Development Lifecycle (AI-DLC) guide. - AWS CloudFormation User Guide. - Amazon Athena User Guide. - AWS Secrets Manager User Guide. - Interactive dashboards in Amazon Quick Sight. - REST API documentation for GitHub and GitLab.