Organizations/Enterprises are usually divided in to silos such as Infrastructure, Development, Operations support, Network etc.
Immaterial of the type of organization, User/Consumer/customer experience crosses all silos and in order to provide a rich experience all the silos need to work together. While a clear demarcation between the roles and responsibilities of each silo is absolutely needed, the change in technology landscape, viz. Virtualization, and cloud has enabled a development and operations paradigm shift, which requires us to rethink/redefine these silos. The feasibility to AUTOMATE in general creates and enables that paradigm shift blurring the lines between the silos. While this paradigm screams collaboration, DevOps in my view is much more.
Enterprise Standard practices such as ITIL, COBIT distinguishes the following areas:
Plan, Build & Provision, Run
Plan & Organize, Acquire and Implement, Delivery & Support, Monitoring
Most IT organizations structures around those areas and create divisions/department/silos with shared functional responsibilities. Traditionally separate responsibilities have now started to overlap due to automation and virtualization.
In an attempt to elucidate this paradigm shift and create a clearly defined process, responsibility delineation, with increased extended cross department collaboration a moment/practice was necessary, and thus DevOps was born.
Definition: My version I
DevOps can be defined as a collaborative practice of development, automation, support and Integration, between traditional Silos/departments extending the entire application lifecycle resulting in a change of Enterprise culture and accelerated delivery of business value
Wikipedia refers to DevOps as a clipped compound of “development” and “operations”, before delving deeper let’s look at the activities performed by traditional silos.
Operations traditionally get involved with the deployment/Release management, basic testing, Configuration, Installation, support, Server updates, monitoring and maintenance, Application maintenance, such as database backups, Batch scheduling, occasionally application monitoring infrastructure management and provisioning, security management and provisioning all in the context of Production environment and in a few cases Production like Pre-Production environments as well.
From a development teams stand point Application/Product Lifecycle involves the following:
Application development, Project Management, Program Management, Testing – Performance, Unit, Functional and more, Release scripts, Build Tools, Code Insight, Code Review, Deployment Management, Dependency Management, Static Analysis, Version Control along with some shared responsibilities on the operations and support side.
When it comes to troubleshooting applications there is usually a blame ping pong between teams, and so it is when it comes to delay in projects.
While, at the end of the day for a successfully run organization there has to be a strong collaboration. This is where DevOps as a practice thrives.
A single team represented by the members of various silos working as one. The state which I call Nirvana.
Definition: My Version II
DevOps = 1 Agile/Lean team + (Continuous Integration + Continuous Deployment) across all environments + Automated Testing + Continuous measurement and adaptation + Monitoring + Support + Exec & stakeholders buy in.
Please note that the above definition applies to production also, and not just non production environments.
There are agile believers and there aren’t. The age old practices takes long enough to change, and can ONLY come with a change in culture and complete buy in. Same applies to DevOps, and all the more.
Fret Not !, When the change happens it comes with the following benefits
- Increased collaboration between departments (A primary success factor)
- Overall cost reduction due to automation
- Increased frequency of deployments, faster time to market
- Optimized support and maintenance
- Application built for performance
- Reduction in time spent fixing – Huge Operational gain
- Cross functional teams
Cautionary Note: While DevOps adoptability has increased considerably, some see it as a clichéd term
Following are some of the DevOps Obstacles:
- Change in role and permissions results in security and compliance concerns
- Leaders who want to create a business case for DevOps adoption, find ROI measurement and justification difficult.
- Larger Organizations with well-defined organizational structure and the overall complexity in making the change happen
- Lack of clearly defined R & R
- Lack of Leadership support
- Lack of skills
Since DevOps covers too much ground, there isn’t one tool which covers all the functionality.
I have documented lot of tools in my earlier blog, which are all part of the application lifecycle: https://prashanthpanduranga.wordpress.com/2015/04/22/architecting-extremely-large-scale-web-applications-a-must-read-for-every-architect/
Note: Some of the tools may serve multiple areas, I have categorized them under what I consider their primary area
Virtualization and Containerization tools
|Hyper-V, VMWare, KVM, Xen, VirtualBox, Vagrant, Docker, Boot2Docker, WAMP, MAMP, LXC, Solaris Containers, OpenVZ, Warden, BSD Jails, V-Server|
|Cobbler, FAI, Kickstart, Spacewalk, OpenQRM, PXEBoot, Jeos, BoxGrinder, Eucalyptus, AppLogic|
Configuration Management, Deployment
|Puppet, MCollective,, Chef, Ansible, CFEngine, Saltstack, RANCID, Ubuntu Junu, Capistrano, Salt, Crowbar, Pallet, CloudFormation, Escape, ConfigGen|
Test and Build Systems
|Solano, Jenkins, Maven, Ant, Gradle, Hudson, Bamboo, TeamCity, Travis, NAnt, MSBuild, Gant, Make, SauceLabs, webrat, Cucumber, jbehave, jasmine, testing, selenium, JMeter, Neoload, loadrunner, BlazeMeter, qtp, protractor, webload, Gomez, cactus|
|New Relic, Nagios, Icinga, Graphite, Ganglia, Cacti, PagerDuty, Sensu, Monit, runit, Supervisor, GOD, Bluepill, UpStart, system|
|PaperTrail, Logstash, Loggly, Logentries, Splunk, SumoLogic|
|Snorby Threat Stack, Tripwire, snort,|
|Ivy, Nexus, Archiva, Artifactory, Bundler|
|Code Review, Code Insight||Crucible, Gerrit, FishEye, Stash|
Version Control and IDE
|Git, Perforce, TFS, Subversion, SVN|
|Database Change Management||Liquibase, Flyway, dbdeploy|
|Collaboration tools/Issue Tracker||Confluence, Jira, SharePoint, Bugzilla, redmine, rally, GreenHopper|
|IDE||Visual Studio, Andriod Studio, Eclipse, Blend, Dreamweaver, Expression Studio, gcc, IDEA, MonoDevelop|
|Static Analysis||FXCop, CheckStyle, Clover, cobertura, chimera, unibeast,|
Note: This is NOT a comprehensive list of tools.
There are lot of great tools along with reference architecture in the following link:
I guess it is not fair to call something a state of nirvana, without recommendations on how to get there. So here it is:
- BUY-IN. Any culture change needs complete buy-in from management with reinforcements through strong leadership to make it happen
- Perform an Enterprise Architecture assessment (This will help you understand the maturity of the organization and highlight the areas to work on)
- Revisit your operational Model (This will need Tweaking) and create a roadmap for Target Operational Model
- Create a DevOps roadmap
- Create a CMDB, if there already isn’t one. There are other tools that are important, CMDB tops the list.
- Tear down silos. Everybody is responsible equally for a quality delivery. Clear strategy on how the individual units will work in collaboration is mandatory. Layout out the modified R & R for the departments. (DevOps doesn’t mean that Operations/Admins doesn’t need to be there anymore, they are absolutely needed, and the mode they operate in will vary)
- There might be changes needed on the infrastructure management contracts if it is managed by third party. If they are already tied in to your processes for Infrastructure automation then there will be no modifications required, certain contracts even impose when you can deploy and when you cannot, those will need to be changed. One of the most important steps for a successful DevOps is your ability to automate the infrastructure and hence the need for greater attention
- You will start your journey towards a Release management 2.0. Company size however small or big, Release windows to Trains with varying cadence, this journeys destination is a fully automated deployment with automated testing.
- Get ready to be an automated testing company. Manual Testers keen to give their scripting skills a makeover, here is your time.
- Transparency, Transparency and Transparency from development to dash boards, to code coverage to monitoring to performance, COMPLETE transparency across departments
- Feedback, Measurement and adaptation.
DevOps Nirvana was never meant to be easy, and isn’t an end state, it is a beginning of continuous Improvement.
About the Author:
Prashanth B Panduranga
DevOps Image: The DevOps image is a combination of the following images. Combination done by me.