localhost Challenges when Scaling: Continued adventures in Swift's sharding

Matthew Oliver
http://linux.conf.au/schedule/present...
Background: Swift is an open-source, highly available, distributed, eventually-consistent object storage system. As the project has matured, so has the need to store more and more data. Some sites have moved to using expensive solid state drives (SSDs) for storage, but there has to be a better way we can solve this in software, right?

This presentation continues our Swift sharding adventure from where we left it last year - but now addressing a new set of challenges we've come across in our proof of concept container sharding implementation. During testing of even larger data sets we've come across a number of limitations of the underlying SQLite database, meaning we've had to develop work arounds so that we are able to shard even larger containers. Come and see how we performed large scale testing, how we narrowed down where the limitations were, how we developed our solutions to these problems, all the while trying to maintain backwards comparability.

localhost Challenges when Scaling: Continued adventures in Swift's sharding
3 Likes	3 Dislikes
88 views views	3.5K followers
Education Creative Commons Attribution licence (reuse allowed)	Upload TimePublished on 19 Jan 2017

Related keywords

lcare mexicana,camara de diputados,lca uanl,lcad,lca significado,lca virtual,lca arquitectos,lcai,local,l'carnitina,lca capital,lca pais,lca abogados y consultores,lcare mexicana s de rl de cv,lcat,alcatel,lcase,lcase vba,calidad,