Optimizing global portal databases helping retailers and cities to monitor the recycling of 50 billion used bottles and cans a year in the world.
Optimizing global portal databases helping retailers and cities to monitor the recycling of 50 billion used bottles and cans a year in the world
TOMRA
CUSTOMER SPOTLIGHT
The company’s solutions collect aluminum, plastic and glass beverage containers to be continually reused and recycled back into new bottles and cans, particularly in regions with deposit return schemes. With approximately 85,000 installations across more than 60 markets, TOMRA reverse vending machines capture over 46 billion used bottles and cans each year. This reduces reliance on raw materials, and ensures fewer containers end up in our streets, oceans and landfills. TOMRA’s reverse vending machines, digital solutions and service make recycling easy for the industry, system owners, retailers and consumers to contribute to a world without waste. Visit our Reverse Vending pages on tomra.com/collection.
Project Challenge
Due to a cyber-attack on AWS platform and due to the fact that their main clients are retailers, Tomra collection decided in 2023 to move their full stack to Google Cloud. The new stack needed to follow very stringent security rules and limit the downtime. On top of it, Tomra wanted a managed platform that greatly reduces the total cost of ownership for their very large databases (about 100 TB of SQL server databases in the US, Australia and Europe). Finally Tomra were unable to manage indexes (too large), data corruption and data maintenance (purge) as the activity inserted more rows faster than the deletes.
Solution
Phase 1: The escape
Tomra Collection decided to quickly in 36 hours lift and shift their database infrastructure to GCP. CHDS accompanied the client to properly size the infrastructure and install properly the database. At the same time, we reduced considerably the technical debt by upgrading the version from 2019 to 2022 and enforce encrypted connections. As we had to deal with large databases, over 20TB, we build a solid partner interconnect and multithread backups by striping them and then set a log ship mechanism onto GCP infrastructure, reducing considerable downtime duration. CHDS helps Tomra being funded by Google Cloud for the migration, decreasing considerably the migration cost.
Phase 2: The re-platforming
Tomra Collection has to deal with many environments and many databases across all continents. Thanks to their cloud native infrastructure using Cloud PubSub, Cloud run and BigQuery, Tomra was able to integrate nicely the whole data cycle with Cloud SQL. This phase was not done in a hurry and properly to optimize performance, reduce licence cost and properly set up the right maintenance solution. Disconnected read replicas were used to provide on-demand snapshots to the BI reporting in a few minutes, despite the large database footprint. To ensure security, we use SSL, cloud Sql proxy and the necessary database flags and labels. Monitoring was handled with cloud logging, BigQuery sink, system and query insights. The team uses terraform for Infrastructure as code and flyway for database changes deployment. We were also able to downgrade the edition from Entreprise to standard for SQL server decreasing greatly licence cost. We kept large machine type without increasing the visible cores by using simultaneous multithreading to off providing better machines both on compute and iops for a decreased price.
Phase 3: The Optimization
Tomra collection needed better performance and better retention management on cloud SQL. CHDS implemented specific data partitioning to improve the management and the performance of the largest tables, removing the issue of index and data retention management. CHDS also migrated all binary contents onto Google Cloud storage, greatly decreasing the disk footprint. Further tuning were done for the reporting and for the transactional queries stabilizing the non-peak time to 5% CPU and peak time to 45%. The elasticity of the compute allows the Tomra Collection to decide how to proceed considering the downtime for the instance compute resource to scale up or down.