{"repo":"hellige/au","free":true,"listed":false,"github":"https://github.com/hellige/au","clone":"git clone https://github.com/hellige/au.git","description":"Binary JSON encoder/decoder, library and tool, mostly for log files","language":"C++","stars":25,"topics":["json","logging","tools"],"license":"MIT","category":"cli-tools","readme_excerpt":"au is a file format, header-only C++ library and command-line tool for working with sequential record-oriented data, primarily log files. The tool supports grepping in .au files, and for convenience supports grepping normal JSON files and plain timestamped ASCII log files as well. The combination of binary search within large files, and the ability to index and offer random access to gzipped files, make this a very useful tool even for plain ASCII logs! A rust port of the encoder/decoder library is available here. Motivation and usage Ok, so you're doing some logging. The records have some structure but it's ragged and irregular, like maybe every line has a timestamp and a couple of other fields, but beyond that different kinds of events have different fields. So you decide to use JSON. But now your files start getting really big, the key names are duplicated all over the place, you have lots of huge ASCII timestamps all over the place, and it all feels pretty wasteful. Your program is also spending lots of time formatting JSON, which feels pretty wasteful as well. But there are a few things that you don't want to lose: - structured but schemaless: the records are nicely structured but also self-describing. You don't need to update a schema in order to add a new record type or otherwise rearrange the content of your logs. - greppable, tailable: you can use unix tools to inspect these files in ways normal for log files, particularly being able to tail the end of the file witho","default_branch":null,"files":null,"tree":[],"storefront":"/r/hellige","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/hellige/au/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}