{"repo":"wolfeidau/aws-billing-store","free":true,"listed":false,"github":"https://github.com/wolfeidau/aws-billing-store","clone":"git clone https://github.com/wolfeidau/aws-billing-store.git","description":"This project automates setup of Cost and Usage Reports (CUR) in a billing account with an Athena table enabling querying of the latest data for each month.","language":"Go","stars":13,"topics":["athena","aws","billing","datascience"],"license":"Apache-2.0","category":"auth-billing-email","readme_excerpt":"aws-billing-store This project automates setup of Cost and Usage Reports (CUR) in a billing account with an Athena table enabling querying of the latest data for each month. This is an alternative to the pre configured athena setup provided by the Billing Team and is more suited to customers with 10s or 100s of CUR files provided in each report. Overview In summary this project deploys: 1. Creates the CUR in the billing service and the bucket which receives the reports. 2. Configures a Glue Database and Table for use by Athena. 3. Deploys a lambda to manage the partitions using eventbridge S3 events. Once deployed all you need to do is wait till AWS pushes the first report to the solution, this can take up to 8 hours in my experience, then you should be able to log into Athena and start querying the data. Why? The goal of this project is to provide a consistent view of cost and usage reports (CUR) at all times in a Athena table. To do this we use hive symlinks, which are updated each time a new snapshot arrives providing an atomic single file update for new CURs. This is different to the status table provided by AWS Billing team in the pre configured Athena setup. How It Works When you enable the option to keep all versions of the CUR, AWS will upload a new snapshot then once complete update the manifest containing a list of file paths for that billing period. We use an s3 file create event to trigger reading of that manifest and creation of a symlink in the hive directory we","default_branch":null,"files":null,"tree":[],"storefront":"/r/wolfeidau","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/wolfeidau/aws-billing-store/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}