A few months ago, the Ruby Central team, inspired by our work on ClickPy, asked if we could load public RubyGems download stats into a public ClickHouse instance. The goal was to make it easier for the open-source community to analyze the data. We were glad to help, supporting open source projects has always been a priority for us, and we enjoy working with large, real-world datasets that show what ClickHouse can do.
After bringing the dataset into our SQL Playground, we decided to go a step further and build a dedicated analytics site for the RubyGems community.
Today, we're introducing ClickGems, a new platform that provides analytics for RubyGems. It lets you explore download statistics for more than 200,000 gems, covering data from 2017 to today and totaling over 200 billion downloads.
What is ClickGems? #
ClickGems is a website designed to empower Ruby developers with in-depth analytics for their favorite Ruby gems.
ClickGems follows the same purpose as ClickPy, so we decided to reuse the same code base. For now, development lives in a dedicated branch of the ClickPy GitHub repository, though we may fork it later if it makes sense.
If you've used ClickPy before, ClickGems will feel familiar. The design and features are the same. The landing page shows global statistics such as recent releases for popular gems and top repositories.
From there, you can click on any gem to open its detail page. Each gem page includes charts and metrics such as downloads over time or downloads by country.
How we build visualizations #
Each visualization is powered by a SQL query, which you can view by clicking the link icon in the top-right corner. This opens the SQL Playground, where you can see the query and run it against the raw data. You can use this as a base to build your own visualizations.
To ensure fast execution, we use Materialized Views in ClickHouse to pre-aggregate data as it's ingested. For more details on how we set up these Materialized Views to speed up queries for ClickGems, see our previous blog post on bringing RubyGems data into ClickHouse.
New feature: Shareable charts #
While building ClickGems, we had the chance to work with Metabase, which offers the ability to host and embed charts anywhere. Thanks to their support, we were able to add this functionality to both ClickGems and ClickPy.
Now, from any gem or package page, you can export a chart and embed it directly into your website, documentation, blog post, or project page. This makes it easy to share usage trends and showcase the impact of your favorite libraries.
How we power ClickGems #
ClickGems and ClickPy, along with other public demos like our SQL Playground and AgentHouse, are all powered by the same ClickHouse Cloud deployment.
This unified infrastructure allows us to manage large volumes of data and use cases while keeping the infrastructure and maintenance cost fairly low. ClickHouse ability to ingest large volumes while handling large numbers of parallel queries makes it possible.
We also decided to leverage compute-compute separation within our ClickHouse Cloud in order to dedicate resources to ClickPy and ClickGems website. This architectural decision ensures that peak usage on one service, such as a surge in ClickGems queries, does not affect the performance or availability of other services like ClickPy or our other public demos. This guarantees a smooth and consistent experience for all our users across all our analytics platforms.
Below is a high-level overview of the deployment.
Try ClickGems! #
This is exciting to see three organizations, ClickHouse, Ruby Central and Metabase come together to offer a fully fledged analytical application to the open-source community!
We invite you to go today to ClickGems and see how your favorite gems are doing. We'd also love your feedback and contributions, feel free to open an issue in the ClickPy GitHub repository if you have questions, suggestions, or ideas.
