Tuesday, 14 May 2013

Continuous delivery using MongoDB

Introduction


Last weekend I organized, managed and supported our production release for the first time. Our current primary data store is SQL Server and the release follows roughly this order:
  • Shutdown application
  • Backup databases
  • Run data definition deployment scripts and data scripts
  • Deploy the application
  • Start-up application
  • Test application

From shutdown to start-up there are a good few hours off downtime for the application. That's why releases usually happen on the weekend. It would be nice however if we would be a bit more flexible and the team would appreciate it if we could do deployments after hours during the week. To be able to do this though we would need to cut down the deployment time.

One way of doing this is to use a schemaless database like MongoDB. Since MongoDB doesn't enforce a schema there are no data definition scripts to run either on the database side. The schema is enforced on the application level. That means that the application is responsible for writing data in a safe way and providing read methods which can retrieve the stored data again. By deploying the new version of the application we would therefore also roll out the new version of the schema implicitly.

Data migration in MongoDB

After deploying the new version of the application existing data needs to be migrated. Lets take for example the common case of adding a new field to a collection. In RDBMS we would add the column using a DDL statement and either set a default value or run a batch update to populate the column.

In MongoDB there is a incremental way of achieving the same thing. Currently I am reading the book "NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence" by Pramod J. Sadalage and Martin Fowler which introduced an interesting pattern to achieve incremental database deployment.

  


The application needs to make sure that during the transition phase it can read both the old version of the document and the new version. However once the document gets saved the application would save only the new version. That way the entire data set will get migrated over time. To implement this the book suggest to add a field schema_version to each document. This schema version could correspond to the version of the application. Based on this field the application can decide if a document has been migrated already. If a document is loaded which is of an older version the application can provide special code to execute for migrating to the next version and save it.

Once all documents are migrated the migration code in the application can safely be removed in the next release.



18 comments:

  1. "Nice and good article.. it is very useful for me to learn and understand easily.. thanks for sharing your valuable information and time.. please keep updating.php jobs in hyderabad.
    "

    ReplyDelete
  2. Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging. If anyone wants to become a Java developer learn from Java Training in Chennai. or learn thru Java Online Training India . Nowadays Java has tons of job opportunities on various vertical industry.

    ReplyDelete
  3. I appreciate that you produced this wonderful article to help us get more knowledge about this topic.
    I know, it is not an easy task to write such a big article in one day, I've tried that and I've failed. But, here you are, trying the big task and finishing it off and getting good comments and ratings. That is one hell of a job done!



    Selenium training in bangalore
    Selenium training in Chennai
    Selenium training in Bangalore
    Selenium training in Pune
    Selenium Online training

    ReplyDelete
  4. Nice Information
    "Sanjary Academy provides excellent training for Piping design course. Best Piping Design Training Institute in Hyderabad,
    Telangana. We have offer professional Engineering Course like Piping Design Course,QA / QC Course,document Controller
    course,pressure Vessel Design Course, Welding Inspector Course, Quality Management Course, #Safety officer course."
    Piping Design Course in India­
    Piping Design Course in Hyderabad
    Piping Design Course in Hyderabad
    QA / QC Course
    QA / QC Course in india
    QA / QC Course in Hyderabad
    Document Controller course
    Pressure Vessel Design Course
    Welding Inspector Course
    Quality Management Course
    Quality Management Course in india
    Safety officer course

    ReplyDelete
  5. Thanks for sharing
    Yaaron Studios is one of the rapidly growing editing studios in Hyderabad. We are the best Video Editing services in Hyderabad. We provides best graphic works like logo reveals, corporate presentation Etc. And also we gives the best Outdoor/Indoor shoots and Ad Making services.
    video editors studio in hyderabad
    short film editors in hyderabad
    corporate video editing studio in hyderabad
    ad making company in hyderabad

    ReplyDelete
  6. Thanks for sharing this. Get all the replays and videos of pionoy channel free without getting any paidmembership. All the replays are avaialble on pioy tambayan network

    ReplyDelete
  7. thanks for sharing this tropics.
    AWS Solutions Architect Training in Bangalore
    https://onlineidealab.com/aws-training-in-bangalore/

    ReplyDelete
  8. uv gullas collge of medicine MBBS in Abroad

    There is an opportunity for many students to become a doctor. The rise in the percentage annually of students is indicative of the increased interest of students in the medical sector. But the known fact is that it involves a lot of pressure and money to become a doctor. That is because there are fewer seats available in UV Gullas College of medicine. Students who are interested to become doctors will either enter the Private Medical College or simply prefer studying abroad when the rigorous tests are not taken. We will discuss the choices for Indian students for MBBS abroad.

    Eligibility Requirement MBBS in Abroad

    * The student must be 17 years plus and the admission is being taken.
    * The student must have completed the class 12th.
    * If the candidate is in the overall classification, the applicant must be aggregated with at least 50% in physics, chemistry, and biology and must be required to speak English.

    visit : http://uvgullascollegeofmedicine.in/
    contact : 9444666890

    ReplyDelete
  9. embedded project centers in chennai Then things that are too hard or too expensive to do can be programmed using conventional technologies such as logic, time switch and so on. Link such an intelligent product to the Web
    and more can be done.

    ReplyDelete
  10. It is amazing and wonderful to visit your site.Thanks for sharing this information,this is useful to me...
    http://traininginmedavakkam.in/training-courses/sap-abap-training-in-medavakkam/
    http://traininginmedavakkam.in/training-courses/ccnp-training-in-medavakkam/
    http://traininginmedavakkam.in/training-courses/revit-architecture-training-in-medavakkam/
    http://traininginmedavakkam.in/training-courses/digital-marketing-training-in-medavakkam/
    http://traininginmedavakkam.in/training-courses/cad-training-in-medavakkam/
    http://traininginmedavakkam.in/training-courses/sap-fico-training-in-medavakkam/

    ReplyDelete
  11. Great blog !It is best institute.Top Training institute In Coimbatore
    http://glimtechnologies.com/aws-training-coimbatore/
    http://glimtechnologies.com/angular-js-training-coimbatore/
    http://glimtechnologies.com/dotnet-training-coimbatore/
    http://glimtechnologies.com/dotnet-training-coimbatore/
    http://glimtechnologies.com/sas-training-coimbatore/
    http://glimtechnologies.com/wordpress-training-coimbatore/

    ReplyDelete
  12. Great blog !It is best company for online job support from India
    http://onlinejobsupport.net/job-support/openspan-online-job-support/
    http://onlinejobsupport.net/job-support/uipath-online-job-support/
    http://onlinejobsupport.net/job-support/automation-anywhere-online-job-support/
    http://onlinejobsupport.net/job-support/salesforce-admin-online-job-support/
    http://onlinejobsupport.net/job-support/microsoft-azure-online-job-support/
    http://onlinejobsupport.net/job-support/data-science-online-job-support/
    http://onlinejobsupport.net/job-support/machine-learning-online-job-support/

    ReplyDelete
  13. The information you have posted is very useful. I really enjoy reading and also appreciate your work. Please keep inspiring more readers!

    Android app development
    iOS App Development
    Freelance software developer

    ReplyDelete
  14. This blog very easily understandable. Thanks for sharing such an informative post with us. This is a nice post in an interesting line of content.
    MongoDB Training in Bangalore

    ReplyDelete