DevOps

From BC$ MobileTV Wiki
(Redirected from DevOps metrics)
Jump to: navigation, search
DevOps.png

The union of and full cooperation between Development (Dev) teams/departments and Operations (Ops) teams/departments within (and potentially even across partner) organizations, towards realization of a common set of corporate and/or societal goals, over the entire service lifecycle of systems, from design through the development process to production support.

DevOps evolution.jpg


Metrics

The DORA report has identified the following 4 metrics as the most important for measuring the health and/or success of any DevOps initiative:

  1. Lead Time - a key metric on which to base our improvement efforts, striving to reduce the time between ideation and delivery into customers' hands
  2. Deployment Rate - these should be constant pace and typically striving for shorter time deltas between each release, meaning more releases is viewed as an indicator of good team/project/product health
  3. Change Failure Rate - depending on organization, in addition to the typically measured rollbacks this could include any unplanned or partial outages in its measurement, but is distinct from "Defect Escape Rate" and is not meant to be a catch-all for any and all imperfections making it to PROD... the idea is to measure any unexpected Business Value lost while introducing new Business Value)
  4. MTTR - how long it takes to recover from any failures or major problems in any given environment (but with a strong if not obvious emphasis on Production), and how gracefully we do so, is an indicator of our team/project/product's capacity to deliver more value faster with high confidence in our abilities to work through any unforeseen issues; we aim to minimize the amount of time between an issue introduced an a corresponding hotfix or full release to address it (and as per "Change Failure Rate" metric which comes first, addressing any such failures or problems should not introduce other issues)

For more metrics, see: Optimization#Metrics

[1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]

Techniques

Digital Transformation

Digital Transformation (commonly abbreviated as either DT or DX; sometimes also commonly referred to as "Digital Experience") is the use of new, fast and frequently changing digital technologies to solve problems. In the enterprise setting that's business problems (such as how to drive one or more key metrics like revenue, increase market share, customer loyalty, etc). Regardless of metric(s) chosen to measure a Digital Transformation, its successful adoption and continued implementation typically hinges on how well the initiatives chosen tie to the bottom-line and increase the health and modernization of the business.

Although Digital Transformation may take many forms, a key theme is switching a majority of the applications and tools used from expensive highly customized to Commercial Off-The-Shelf (COTS) and/or Software-as-a-Service (SaaS) where practical. This is not merely "for the sake of being Cloud/SaaS-focused", but because often (but certainly not always) these solutions can be more cost-efficient than a sprawling array of various in-house, or heavily customized self-hosted software and tools. Customized in-house solutions (where they do persist) should be focused on differential capabilities, business logic, and unique product/service offerings not available on other platforms or combinations of platforms.

Tools and custom internally used "business-supporting" applications aside, the final and most important piece of the Digital Transformation puzzle is to convert a certain amount of the key Lines of Business (LoB) over to higher Return On Investment (ROI) cost-per-sale and cost-per-acquisition digital technologies rather than relying entirely on the limitations of any current physical markets (i.e. B2C Retail, or, C2C goods/materials exchange). This could mean creation of an E-Commerce/E-Business enhanced Website, WebApp, MobileApp or some other mechanism for the digital exchange of goods and/or services to supplement traditional physical channels (especially in channels where there was no prior online, mobile or virtual presence), introducing such, as an increasingly viable alternative for exchange of the goods/service.

DigitalTransformation.png

Digital Transformation is frequently touted as a critical factor of what's being dubbed the "4th Industrial Revolution", with many analysts predicting that only businesses who succeed in their Digital Transformation efforts will be around in that vision of the future.

Some executives and consultants like to list the following criteria for gauging a company's "Digital Transformation maturity":

  1. Market agility
  2. Delivery velocity
  3. Frictionless personalized experiences
  4. Experts in analytics, integration & security
  5. Invest in and realize benefits from robotic process automation

Although, without specific OKRs/KPIs/KPMs these are difficult intangibles to measure.

[13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42]

Business Value Baked-In

Also known as "DevBizOps". This implies that each change or feature requiring any substantial development effort should have a specific Business Value estimation tied to it, not just a Story Points or other form of effort estimate as per typical Agile development. We then measure the change and how it has impacted your Key Performance Indicators (KPIs) towards reaching your Objectives & Key Results (OKRs).

DevBizOps.png

For more specifics on this, see: Business Value

[43]


Quality Baked-In

Also known as "DevTestOps".

DevTestOps.png

For more specifics on how to achieve this, see: Continuous Testing [44] [45]


Security Baked-In

Also known as "DevSecOps".

DevSecOps.png

For more specifics on how to achieve this, see: Security [46] [47] [48] [49] [50]


Risk Management

NIST RiskManagement-framework.jpg

[54] [55] [56] [57] [58] [59] [60] [61]

Change Management

ITIL DevOps ChangeManagement.png

[62] [63] [64]


Release Management

ReleaseManagement vs EnvironmentCoordination.jpg

[65] [66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79]

Cultural Patterns

DevOps culture can be summarized with the following diagram:

DevOps culture.jpg

[80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90]

Pre-Mortem

Risk evaluation approach/meeting where any higher risk or even potentially complex Backlog items such as high-level feature requests or User Stories (or if still receiving them then converting to Stories, Requirements Documents) are identified, voted on and/or ranked and planned out in terms of which may present blocking dependencies or which may cause the team the most trouble within the desired timelines or current technology architecture/stack.

Blameless Post-Mortem

[91]

[92] [93] [94] [95] [96] [97] [98] [99] [100] [101]

CALMS

DevOps adoption and rollout strategy.

DevOps CALMS.png
  1. Culture
  2. Automation
  3. Lean
  4. Measurement
  5. Sharing


[107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118] [119] [120] [121] [122] [123] [124] [125] [126] [127] [128] [129] [130]

INVEST

Agile-UserStores-INVEST.png

The INVEST "mnemonic" is a guideline for clear and actionable User Stories and all Product Backlog Items (PBI) in general, which ensure that once picked up there will be minimal "hand-offs" or "wait time".

  • I=Independent
  • N=Negotiable
  • V=Valuable
  • E=Estimable
  • S=Small
  • T=Testable


[131] [132] [133] [134] [135] [136] [137] [138] [139][140] [141] [142] [143] [144] [145]


The 3 Ways

Three core principles that promote DevOps. These should be considered the steps to take when analyzing a company, department, team or product level situation for best approaches to take to bring about the types of changes required to reach the ideals of DevOps. While not specific to DevOps "The 3 ways" can be applied for the sole goal of realizing operational efficiencies and/or delivery effectiveness, the need to adopt some or all DevOps concepts in the process will vary to the degree that a given Line-Of-Business (LoB) depends on Software and technology in general. This dependence has grown quite large in the modern world, but will certainly vary from industry to industry, company to company, team to team, product to product, service to service.

3ways-DevOps.jpg

  1. Flow
  2. Feedback
  3. Continual Experimentation & Learning

[146] [147] [148] [149] [150] [151] [152] [153] [154] [155] [156] [157] [158] [159] [160] [161] [162] [163] [164] [165] [166] [167] [168] [169] [170] [171] [172] [173] [174] [175] [176] [177] [178] [179] [180] [181] [182] [183] [184] [185] [186] [187] [188] [189] [190] [191] [192] [193] [194] [195]


The 5 Ideals

In "The Phoenix Project", author Gene Kim introduced the concept of the "Three Ways", and, the Four Types of IT Work. In his follow-up book "The Unicorn Project", readers are taken through The Five Ideals, or concepts that help drive significant and positive change within the Business and IT.

These include:

  1. Locality and simplicity -- don't underestimate the significance of smaller-scale changes to a single software or infrastructure component.
  2. Focus, flow and joy -- provide quick feedback on code so developers can stay focused and enjoy their day-to-day work.
  3. Improvement of daily work -- implement systemic fixes to streamline daily tasks.
  4. Psychological safety -- ensure all staff feels comfortable and empowered to share their opinions openly.
  5. Customer focus -- instill a culture where leaders across all functional silos focus on what the organization as a whole, and its customers, need long-term.

[196] [197] [198] [199] [200] [201] [202] [203] [204] [205] [206] [207] [208] [209] [210] [211] [212] [213] [214] [215] [216] [217] [218] [219] [220] [221] [222] [223] [224]


FLOW framework

The FLOW framework:

FLOW-framework.png

It focuses on prioritizing the 4 separate types of backlog item (which can each come from a number of different sources from customer feedback/demand, to regulator/legislative obligation, to business unit request, to personally direct-requested, to team internally-driven ideation or gap identification):

Backlog 4-types.jpg

They can be described as follows:

Item Delivers Primary Sources Goal Artifacts
Feature New/more business value Pushed by Customers (or Business ideation) Additional service offerings & revenue streams Epic, User Story, Functional Requirement
Defect Quality Pushed by Business (on behalf of Customers) Improved customer experience & higher engagement Bug (PROD-related), Defect (project/env-related), ITIL incident
Risk Attack vector eliminated, Governance compliance met Security and/or Risk & Compliance dept, External Auditors Data privacy & service reliability Vulnerability patch, Regulatory criteria, Non-Functional Requirement
Debt Removal of impediments to future deliveries Internal Architects and/or Front-line workers (those delivering) Operational efficiency & reduced Total Cost of Ownership (TCO) Refactored code, Automated infrastructure, Process improvement

[225] [226] [227] [228]

Infrastructure-as-Code

IaaC-vs-IaaS.png

For more, see: IaaS

[229] [230] [231]

SRE

Site Reliability Engineering (SRE) is a particular practice or implementation of DevOps that emphasizes:.

  1. Small, testable, measurable, incremental, continuous improvements
  2. Automation for efficiency's sake (not automation's sake)
  3. Practice observability through monitoring vital signs of the entire system, including the four "golden signals": latency, traffic, errors, and saturation
  4. Alerting & Reporting "chain of command" & "responsibility schedules"

SRE-overview.png

[235] [236] [237] [238] [239] [240] [241] [242] [243] [244] [245] [246] [247] [248] [249] [250] [251] [252] [253] [254] [255]

Value Stream Mapping

ValueStreamMapping SOFTWARE.png

[256] [257] [258] [259] [260] [261] [262] [263]

Cultural Anti-Patterns

JDFI Deployments

[264] [265] [266] [267]


Conway's Law

Conway's Law is a hypothesis that: 

organizations which design systems... are constrained to produce designs which are copies of their communication structures

Tragedy of the Commons

The "Tragedy of the Commons" states that if it's everyone's responsibility to maintain the resources, then no one is responsible (or accountable).

TragedyOfCommons.jpg

[268] [269] [270] [271] [272] [273]

Diffusion of Responsibility

Diffusion of Responsibility (also known as "pluralistic ignorance").

BystanderEffect.jpg

Model of intervention and helping to prevent Diffusion of Responsibility: Diffusion-of-Responsibility-intervention-model.jpg

[274] [275] [276] [277]

GroupThink

GroupThink is the effect that a large group can have on decision-making, problem-solving or new idea generation, whereby often the "status quo" or conformist opinions dominate the outcomes due to a fear of ostracism for standing out or demonstrating differing or unfavourable viewpoints.

Groupthink.gif

Some ways to counter GroupThink are:

  • Prevent executives, senior managers and/or bosses from steering the entire conversation or making the decision themselves then managing downwards
  • Invite those external to the problem (other teams), or, third parties (outside the company)
  • Identify outliers or opposing ideas (reverse & inverse)
  • Seek out disagreements
  • Divide by expertise
  • Focus on the right things, same problems
  • Periodically switch roles
  • Come together and discuss
  • Think (individually) and allow time for reflection
  • Decide independently, from a place of informed understanding

Group-Thinking-phenomenon.jpg

[278] [279] [280] [281] [282] [283] [284] [285] [286] [287] [288] [289] [290]


Learned Helplessness

Dev & Ops (silos)

Treating Dev & Ops as two totally separate Teams and/or Departments, with completely separate lines of reporting, performance monitoring, etc.

[291]

DevOops



Resources


Tutorials


External Links


References

  1. ↑ Measuring DevOps Success with Four Key Metrics: https://stelligent.com/2018/12/21/measuring-devops-success-with-four-key-metrics/
  2. ↑ Failure – the fertilizer of "Continuous Learning": https://blog.codecentric.de/en/2020/03/failure-the-fertilizer-of-continuous-learning/
  3. ↑ How to measure the accuracy of forecasts: https://blog.asmartbear.com/forecast.html
  4. ↑ Does Your Kanban System Produce Reliable Probabilistic Forecasts? Thin-Tailed vs Fat-Tailed Cycle Time Distributions: https://getnave.com/blog/thin-tailed-vs-fat-tailed-distribution/
  5. ↑ Kanban Maturity Model – Understanding Lead Time: https://www.kanbanmaturitymodel.com/wp-content/uploads/2020/12/KMM-Understanding-Lead-Time_V1_12282020.pdf
  6. ↑ 5 THINGS YOU NEED TO KNOW ABOUT LEAD TIME (INFOGRAPHIC) https://djaa.com/wp-content/uploads/2021/03/Five-things-you-need-to-know-about-lead-time.pdf
  7. ↑ Latency Numbers Every Team Should Know: https://benjiweber.co.uk/blog/2021/01/23/latency-numbers-every-team-should-know/ (Unit Test runtime, time to CodeReview, PR merge to PROD deploy, etc)
  8. ↑ Top 10 Metrics You Need For Measuring Productivity: https://dzone.com/articles/top-ten-metrics-you-need-to-measure-productivity
  9. ↑ Unlocking Measurability: https://soonersaferhappier.com/install-transparency-unlock-measurability/
  10. ↑ 5 WhitePapers to Improve Software Delivery -- Measure Efficiency, Effectiveness, and Culture to Optimize DevOps Transformation: https://itrevolution.com/five-papers-to-improve-software-delivery/
  11. ↑ A Data Scientist’s Guide To Measuring Product Success: https://shopifyengineering.myshopify.com/blogs/engineering/a-data-scientist-s-guide-to-measuring-product-success
  12. ↑ DevOps Metrics & Measurements Playlist: https://itrevolution.com/topo-pal-metrics-playlist/
  13. ↑ ZDNet -- Digital Transformation Series - The wrap: https://www.zdnet.com/article/digital-transformation-series-the-wrap/
  14. ↑ What does Digital Transformation mean? What are its nine core elements?: https://www.convergehub.com/blog/what-does-digital-transformation-mean-what-are-its-nine-core-elements
  15. ↑ What is digital transformation?: https://www.theguardian.com/media-network/media-network-blog/2013/nov/21/digital-transformation
  16. ↑ What is digital transformation? A necessary disruption: https://www.cio.com/article/3211428/what-is-digital-transformation-a-necessary-disruption.html
  17. ↑ What Digital Transformation will enable: https://www.birlasoft.com/services/digital/transformation
  18. ↑ World Economic Forum (WEF) -- The Fourth Industrial Revolution - what it means, how to respond: https://www.weforum.org/agenda/2016/01/the-fourth-industrial-revolution-what-it-means-and-how-to-respond/ (includes video summary that caused a stir from support to skepticism)
  19. ↑ Davos WEF - Everything you need to know about the Fourth Industrial Revolution: https://www.cnbc.com/2019/01/16/fourth-industrial-revolution-explained-davos-2019.html
  20. ↑ Davos, Blockchain, and the 4th Industrial Revolution: https://media.consensys.net/davos-blockchain-and-the-4th-industrial-revolution-2f3009b68f23
  21. ↑ World Economic Forum -- Leading through the Fourth Industrial Revolution - Putting People at the Centre (2019): http://www3.weforum.org/docs/WEF_Leading_through_the_Fourth_Industrial_Revolution.pdf
  22. ↑ Digital Transformation Report (2018): https://web.archive.org/web/20160314091710/http://altimetergroupdigitaltransformation.com/img/dt-report.pdf
  23. ↑ 5 ways to kickstart a Digital Transformation project: https://webinars.devops.com/five-ways-to-quickstart-a-digital-transformation-project
  24. ↑ What’s driving digital transformation (and what can hamper it)?: https://www.smartinsights.com/manage-digital-transformation/digital-transformation-strategy/whats-driving-digital-transformation-infographic/
  25. ↑ 9 Digital Transformation Statistics That Every Business Should Know: https://www.advance2000.com/digital-transformation-statistics-for-business/
  26. ↑ The Future of Digital Transformation & The Growth of IoT: https://www.utopiainc.com/resources/blog/future-digital-transformation-growth-iot (nice stats/figures on higher profit margins and revenue growth/stability)
  27. ↑ (Maybe Digital Transformation should be called) "Customer Experience lead Transformation": https://advertising3.wordpress.com/2014/04/15/customer-experience-lead-transformation/
  28. ↑ Brian Solis - The Definition of Digital Transformation (INFOGRAPHIC): https://www.briansolis.com/2017/01/definition-of-digital-transformation/
  29. ↑ 6 stages of Digital Transformation in HR (INFOGRAPHIC): https://www.digitalhrtech.com/guide-hr-digital-transformation-hr-transformation/
  30. ↑ CGI UK's take on Digital Transformation (INFOGRAPHIC): https://www.cgi-group.co.uk/en-gb/article/digital-transformation-infographic (generational)
  31. ↑ Benchmarking Digital Transformation to Drive Better Outcomes in Health & Life Sciences (INFOGRAPHIC): https://www.bdo.com/insights/industries/healthcare/infographic-benchmarking-digital-transformation (goals .vs. challenges)
  32. ↑ Digital transformation is disrupting quality assurance: https://www.itworldcanada.com/article/digital-transformation-is-disrupting-quality-assurance-too-capgemini/390418
  33. ↑ 10 digital transformation must-reads: https://enterprisersproject.com/article/2017/7/10-digital-transformation-must-reads
  34. ↑ The Satir Change Model: stevenmsmith.com/ar-satir-change-model/
  35. ↑ Digital Transformation Outlook In 2019: https://www.enterprisemobilityexchange.com/eme-cloud/infographics/digital-transformation-outlook-2019 (what keeps IT Leaders up at night)
  36. ↑ Accelerating Digital Transformation and Application Performance In Turbulent Times: https://www.appdynamics.com/blog/news/accelerating-transformation-now/
  37. ↑ COVID-19 Triggered the Turning Point: https://tasktopblog.wpengine.com/covid-19-triggered-the-turning-point
  38. ↑ COVID-19 is Accelerating the Rise of the Digital Economy: https://www.bdo.com/insights/business-financial-advisory/strategy,-technology-transformation/covid-19-is-accelerating-the-rise-of-the-digital-e
  39. ↑ Your business has gone "Digital" - now what?: https://www.theatlantic.com/sponsored/servicenow-2020/your-business-has-gone-digital-now-what/3400/ (11 best practices for Digital Transformation and measurement)
  40. ↑ The Seven Domains of Product Transformation: https://itrevolution.com/seven-domains-of-transformation/
  41. ↑ What Is Product-Based Software Delivery?: https://itrevolution.com/product-based-software-delivery/
  42. ↑ Digital is not (and never was) an easy button: https://www.mediapost.com/publications/article/373506/digital-is-not-and-never-was-an-easy-button.html
  43. ↑ The Complete List of Resources on Cohort Analysis: https://www.projectbi.net/cohort-analysis-complete-list-resources/
  44. ↑ Will DevOps Culture Come to Smart Buildings?: https://newdeal.blog/will-devops-culture-come-to-smart-buildings-fd00fd88cd04
  45. ↑ Zero Defects -- Baking Quality into the Agile Process: https://www.infoq.com/presentations/Zero-Defects-Baking-Quality-into-the-Agile-Process/
  46. ↑ How NIST CSF and NIST RMF Can Work Together: https://blog.fpmurphy.com/2017/07/how-nist-csf-and-nist-rmf-can-work-together.html#ixzz69Wy0ByZH: https://blog.fpmurphy.com/2017/07/how-nist-csf-and-nist-rmf-can-work-together.html
  47. ↑ DevOps is outdated. Here comes DevSecOps.: https://medium.com/silent-breach/devops-is-outdated-here-comes-devsecops-b98171e541d0
  48. ↑ Achieving DevSecOps with Open-Source Tools: https://www.notsosecure.com/achieving-devsecops-with-open-source-tools/
  49. ↑ Baking Compliance in your CI/CD Pipeline: https://thenewstack.io/baking-compliance-in-your-ci-cd-pipeline/
  50. ↑ Diving into DevSecOps w/ John Willis: https://dzone.com/articles/diving-into-devsecops-w-john-willis
  51. ↑ Scaling a governance, risk, and compliance program for the cloud, emerging technologies, and innovation: https://aws.amazon.com/blogs/security/scaling-a-governance-risk-and-compliance-program-for-the-cloud/
  52. ↑ Assessing Security Controls - Keystone of the Risk Management Framework: https://www.isaca.org/Journal/archives/2016/volume-6/Pages/assessing-security-controls.aspx
  53. ↑ Risk Management Framework (RMF) -- An Overview: https://www.varonis.com/blog/risk-management-framework/
  54. ↑ The Five Rules of Risk: https://www.youtube.com/watch?v=NtX-Ibi21tU
  55. ↑ Risks in IT/Software digest: http://catless.ncl.ac.uk/Risks/
  56. ↑ Managing Risk on Agile Projects with the "Risk Burndown Chart": https://www.mountaingoatsoftware.com/blog/managing-risk-on-agile-projects-with-the-risk-burndown-chart
  57. ↑ 2019 Global Data Risk Report From The Varonis Data Lab: https://info.varonis.com/hubfs/Varonis%202019%20Global%20Data%20Risk%20Report.pdf
  58. ↑ Netflix tech blog -- Open-Sourcing "riskquant" - a library for quantifying risk: https://netflixtechblog.com/open-sourcing-riskquant-a-library-for-quantifying-risk-6720cc1e4968
  59. ↑ Three Innovative Funding Models to Enhance DevOps - IT Revolution: https://itrevolution.com/three-innovative-funding-models-to-enhance-devops/ (incorporating risk factors towards measuring ROI/Budgets)
  60. ↑ A Global Risk Assessment of 2021 and beyond - COVID's impact on national/organizational/personal risk (CHART): https://www.visualcapitalist.com/visualized-a-global-risk-assessment-of-2021-and-beyond/
  61. ↑ Quality Leadership -- Risk Analysis as Quality Advocacy: https://applitools.com/blog/quality-leadership-risk-analysis-as-quality-advocacy/
  62. ↑ Integrating ITIL Change Management and DevOps: https://devops.com/integrating-itil-change-management-and-devops/
  63. ↑ Innovate ITIL -- A DevOps Approach to the ITIL Framework: https://dzone.com/articles/innovate-itil-a-devops-approach-to-the-itil-framew
  64. ↑ DevOps or Die - DevOps and ITSM/ITIL: https://www.slideshare.net/jhbotha/devops-or-die-devops-and-itsmitil
  65. ↑ The Difference Between CI Pipelines and "DevOps Assembly Lines": http://blog.shippable.com/the-difference-between-ci-pipelines-and-devops-assembly-lines-0
  66. ↑ Splunk & VictorOps -- DevOps Release Management Best Practices: https://victorops.com/blog/devops-release-management-best-practices
  67. ↑ Plutora -- Release Management Process and Best Practices: https://www.plutora.com/blog/release-management-best-practices
  68. ↑ Disciplined Agile -- RELEASE MANAGEMENT: https://disciplinedagiledelivery.com/release-management/
  69. ↑ Release Engineering vs. Release Management: https://devops.com/release-engineering-vs-release-management/
  70. ↑ Release Management In DevOps: https://www.softwaretestinghelp.com/release-management-in-devops/ (use of CCB & CAB can still work, with some tweaks)
  71. ↑ Application Release Automation (BEFORE & AFTER): https://flexagon.com/application-release-automation/
  72. ↑ Release and deployment management made easy with 11 ITIL best practices: https://www.manageengine.com/products/service-desk/itil-release-management/
  73. ↑ Intersection of ITIL and Agile methods in the cloud: https://cloudservicemanagementblog.wordpress.com/2016/04/13/intersection-of-itil-and-agile-methods-in-the-cloud/
  74. ↑ ARCAD Release Management Fits With UrbanCode DevOps: https://www.itjungle.com/2015/05/20/tfh052015-story03/
  75. ↑ Enterprise Release Management for DevOps & Continuous Delivery/ From Spreadsheets to Pipelines in Simple Steps: https://www.slideshare.net/xebialabs/from-spreadsheetstopipelinesinsimplesteps
  76. ↑ Why environment provisioning is a key part of DevOps?: https://clarive.com/why-environment-provisioning/
  77. ↑ The Job of the Release Manager: https://www.panaya.com/blog/modern-alm/release-management-process/
  78. ↑ 5 Steps to a Successful Release Management Process: https://www.lucidchart.com/blog/release-management-process
  79. ↑ DevOps Maturity Curve: https://www.slideshare.net/paulpeissner/devops-maturity-assessment-v5
  80. ↑ DevOps Culture -- A Huge Step for Mankind: https://techburst.io/devops-culture-a-huge-step-for-mankind-7bf12db00c43
  81. ↑ 24 Key Capabilities to Drive Improvement in Software Delivery: https://itrevolution.com/24-key-capabilities-to-drive-improvement-in-software-delivery/
  82. ↑ Five cultural changes you need for DevOps to work: https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/digital-blog/five-cultural-changes-you-need-for-devops-to-work
  83. ↑ DevOps Culture - What It Means, Why It Matters: https://www.bmc.com/blogs/devops-culture/
  84. ↑ Shifting Organizational Mindset from “Project” to “Product”: https://medium.com/@ian85smith/shifting-organizational-mindset-from-project-to-product-64e35ede2136
  85. ↑ Shifting Mindsets - How to Break Free From a ´Fixed Mindset´ to a ´Growth Mindset´: https://jodierogers.com/shifting-mindsets-how-to-break-free-from-a-fixed-mindset/
  86. ↑ The five keys to a successful Google team: https://rework.withgoogle.com/blog/five-keys-to-a-successful-google-team/
  87. ↑ What Google Learned From Its Quest to Build the Perfect Team: https://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-quest-to-build-the-perfect-team.html
  88. ↑ Two Simple Questions for Better Business Outcomes: https://itrevolution.com/better-business-outcomes/
  89. ↑ A Visual Guide -- To Sustainable Software Engineering: https://techcommunity.microsoft.com/t5/green-tech-blog/a-visual-guide-to-sustainable-software-engineering/ba-p/2130034?WT.mc_id=green-15747-ninarasi
  90. ↑ Don’t lose developers to bad culture: https://sdtimes.com/softwaredev/dont-lose-developers-to-bad-culture/
  91. ↑ Blameless Postmortem: https://dfrink.com/blameless-postmortem/
  92. ↑ Its not your fault -- Blameless Post-Mortem: https://www.slideshare.net/jhand2/its-not-your-fault-blameless-post-mortems
  93. ↑ How to run a blameless postmortem: https://www.atlassian.com/incident-management/postmortem/blameless
  94. ↑ Blameless culture - post-mortem guidelines: https://postmortems.pagerduty.com/culture/blameless/ (nice summary table on biases that can result in blame)
  95. ↑ 5 Whys — how we conduct blameless post-mortems after something goes wrong: https://medium.com/hootsuite-engineering/5-whys-how-we-conduct-blameless-post-mortems-after-something-goes-wrong-a47687baeacc
  96. ↑ Blameless PostMortems and a Just Culture: https://codeascraft.com/2012/05/22/blameless-postmortems/
  97. ↑ Why Blameless Post-Mortems are Essential: https://victorops.com/blog/blameless-post-mortems-essential
  98. ↑ "Blameless" postmortems don't work. Here's what does.: https://techbeacon.com/app-dev-testing/blameless-postmortems-dont-work-heres-what-does
  99. ↑ Let’s Stop Using The Term “Blameless Culture”: https://www.juliandunn.net/2016/04/07/lets-stop-using-the-term-blameless-culture/ (maybe actionable retros instead)
  100. ↑ "Blameless Post-Mortems" presentation by Jason Hand, DevOps Evangelist, VictorOps (VIDEO): https://www.brighttalk.com/webcast/12395/144639/blameless-post-mortems
  101. ↑ Why Blameless Post-Mortems: https://news.ycombinator.com/item?id=21096653
  102. ↑ What is DevOps?: http://dzone.com/articles/what-is-devops-the-beginners-guide-from-logzio
  103. ↑ What is the "essence" of DevOps?: http://dzone.com/articles/what-is-devops-2
  104. ↑ The Essence of DevOps: http://dzone.com/articles/the-essence-of-devops
  105. ↑ 7 warning signs you need DevOps: http://dzone.com/articles/infographic-7-warning-signs-you-need-devops
  106. ↑ Towards Compliance as Code using the DevOps Audit Defense Toolkit: http://java.dzone.com/articles/towards-compliance-code
  107. ↑ DevOps library: https://www.devopslibrary.com/
  108. ↑ IBM -- Keys to DevOps success (VIDEOS): https://www.ibm.com/cloud/devops/email-containers
  109. ↑ 8 Ways to Track the Success of DevOps Teams: http://dzone.com/articles/8-ways-to-track-the-success-of-your-devops-team
  110. ↑ Blue-green Deployments, A/B Testing, and Canary Releases: http://dzone.com/articles/blue-green-deployments-ab-testing-and-canary-relea-2
  111. ↑ Blue/Green Deployment: http://martinfowler.com/bliki/BlueGreenDeployment.html (the A/B Testing of server infrastructure, test your releases in PROD and quickly revert back)
  112. ↑ Culture, Automation, Measurement & Sharing (CAMS): http://devopsdictionary.com/wiki/CAMS (acronym describing the core values of the DevOps Movement, later expanded to "CALMS" by addition of Lean thinking)
  113. ↑ What Is DevOps?: https://theagileadmin.com/what-is-devops/
  114. ↑ What Is DevOps?: https://techcrunch.com/2015/05/15/what-is-devops/
  115. ↑ Ken Mugrage -- My Definition of DevOps: https://kenmugrage.com/2017/05/05/my-new-definition-of-devops/ (A culture where people, regardless of title or background, work together to imagine, develop, deploy and operate a system)
  116. ↑ What is DevOps and why does it matter?: http://www.zdnet.com/article/what-is-devops-and-why-does-it-matter/
  117. ↑ The benefits of DevOps: https://newrelic.com/devops/benefits-of-devops
  118. ↑ An idiot’s guide to DevOps: https://venturebeat.com/2013/09/30/an-idiots-guide-to-devops/
  119. ↑ Targeting software development to SoCs / NoCs using a DevOps approach: http://devops.com/2014/10/06/targeting-software-development-socs-nocs-using-devops-approach/ (Software-On-a-Chip & Network-On-a-Chip)
  120. ↑ wikipedia: Network operations center (one of two main IT definitions of NOC, the other being Network-On-Chip)
  121. ↑ The Role of a Traditional NOC in the new DevOps World: http://devops.com/2015/02/04/role-traditional-noc-new-devops-world/
  122. ↑ How QA Can Help Lead the DevOps Charge: https://dzone.com/articles/help-wanted-the-pivotal-role-qa-can-play-in-leadin
  123. ↑ DevOps Interview -- Patrick Debois: https://victorops.com/blog/devops-interview-patrick-debois/ (often credited as the creator of the term "DevOps" in 2009 to summarize tendency towards integration of Agile/Scrum/Lean/CI/CD/CT/Automation movements)
  124. ↑ Minneapolis 2014 - Five Years of DevOpsDays: http://legacy.devopsdays.org/events/2014-minneapolis/proposals/five%20years%20of%20devopsdays/
  125. ↑ Why We Need DevOps Now (2012): http://puppet.com/blog/why-we-need-devops-now/
  126. ↑ You, yes YOU -- DevOps' people problem: http://www.theregister.co.uk/2016/01/15/devops_people_problem/ (Chucking a copy of "The Phoenix Project" at the team ain't the answer)
  127. ↑ Inside Obama's "Tech Team": http://gizmodo.com/5960790/inside-obamas-team-tech (with quote from "Head of DevOps, Scott VanDenPlas")
  128. ↑ DevOps At Obama for America & The Democratic National Committee: http://legacy.devopsdays.org/events/2012-newyork/proposals/devopsatobamaforamerica/
  129. ↑ Cultural Challenges of Release Management Best Practices, DevOps Accompanied Obama Campaign Successes: http://puppet.com/blog/cultural-challenges-of-release-management-best-practices-devops-accompanied-obama-campaign
  130. ↑ Culture Automation Lean Measurement Sharing (CALMS) -- A Principle-based DevOps Framework: https://blog.sonatype.com/principle-based-devops-frameworks-calms
  131. ↑ INVEST in Good Stories, and SMART Tasks: http://xp123.com/articles/invest-in-good-stories-and-smart-tasks/ (S=Specific, M=Measurable, A=Achievable, R=Relevant, T=Time-boxed)
  132. ↑ DevOps - The smart person's guide: http://www.techrepublic.com/article/devops-the-smart-persons-guide/
  133. ↑ A DevOps primer -- Start, improve and extend your DevOps teams: http://searchsoftwarequality.techtarget.com/essentialguide/A-DevOps-primer-Start-improve-and-extend-your-DevOps-teams
  134. ↑ Yes, DevOps initiatives sometimes fail: http://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/Yes-DevOps-initiatives-sometimes-fail
  135. ↑ Creating the Safety to Fail: https://medium.com/swlh/creating-the-safety-to-fail-d54798cbbfcd
  136. ↑ The Essence of Mobile DevOps Is to Destroy Silos: http://dzone.com/articles/the-essence-of-mobile-devops-is-to-destroy-silos
  137. ↑ The Basics of Mobile Continuous Integration Workflow: http://bitbar.com/the-basics-of-mobile-continuous-integration-workflow/
  138. ↑ Continuous Integration and Delivery for Agile Teams: https://blog.codeship.com/continuous-integration-and-delivery-for-agile-teams/
  139. ↑ Diving Deeper into DevOps Deployments - Featuring Gene Kim: https://www.youtube.com/watch?v=6nKpKZbvrdM&feature=youtu.be&mkt_tok=eyJpIjoiWXpBNE9ERTBaR1JqWm1FNCIsInQiOiJmbExHRUttaEhFRnJRZm9iQnlxWFB1VnBTOGI5dXVxQ3FBbFBsdlwvVWJZMjk1NGtmeFlDWnNDMk8rMDBnTEJ5bFhVY295R050TGRVb1wvRldXN25EXC9zT3g4VnNXRE9BNGxheVhWS0NSTEdQd1Q4QVdRQkNiMUxoWkd2cm1LWG9RaSJ9
  140. ↑ Atlassian - StatusPage: https://www.statuspage.io/ (communicate with your business/customers about service/website availability providing downtime/uptime transparency)
  141. ↑ WIP -- The System Bites Back: https://medium.com/@johnpcutler/the-system-bites-back-bcf3f16b461
  142. ↑ User stories in an Agile world: https://mohamedradwan.com/2017/09/18/user-stories-in-agile-world/
  143. ↑ INVESTing in good stories: http://msisflotsam.blogspot.com/2014/05/investing-in-good-stories.html
  144. ↑ Product Backlog is DEEP; INVEST Wisely and DIVE Carefully: https://resources.collab.net/blogs/product-backlog-is-deep-invest-wisely-and-dive-carefully
  145. ↑ INVEST in "good" User Stories - Managing Requirements in Agile Business Analysis: https://www.youtube.com/watch?v=G5auvgwTiNk
  146. ↑ Agile and DevOps -- Friends or Foes?: https://www.atlassian.com/agile/devops (includes a nice table summary of the infamous "Three Ways")
  147. ↑ Is DevOps Agile?: https://dzone.com/articles/is-devops-agile
  148. ↑ DevOps -- Where the world meets DevOps: https://devops.com/
  149. ↑ DevOps Reactions: http://devopsreactions.tumblr.com/
  150. ↑ IBM developerWorks -- Learning DevOps: https://www.ibm.com/developerworks/learn/devops/index.html
  151. ↑ Sate of DevOps 2017: https://www.botmetric.com/blog/state-devops-2017/
  152. ↑ DevOps Days: https://www.devopsdays.org/
  153. ↑ What Is DORA and Why Should You Care?: https://devops.com/what-is-dora-and-why-you-should-care/
  154. ↑ The DevOps Assessment: https://devops-research.com/assessment.html (DORA = DevOps xRay Assessment)
  155. ↑ The DevOps Handbook -- An Introduction Summary: https://caylent.com/devops-handbook-introduction-summary/
  156. ↑ DevOps - Boring Is the New Black: https://dzone.com/articles/boring-is-the-new-black
  157. ↑ Microsoft take on question "What is DevOps?": https://www.visualstudio.com/learn/what-is-devops/
  158. ↑ Microsoft Azure - DevOps guides: https://docs.microsoft.com/en-us/vsts/build-release/index
  159. ↑ 10 ways to build highly effective - DevOps teams: https://techbeacon.com/10-ways-build-highly-effective-devops-teams
  160. ↑ DevOps Transformation Using Theory of Constraints - Part 1: https://dzone.com/articles/devops-transformation-using-theory-of-constraints
  161. ↑ DevOps Transformation Using Theory of Constraints - Part 2: https://dzone.com/articles/devops-transformation-using-theory-of-constraints-1
  162. ↑ Definition of what a "Brent" is: http://devopsdictionary.com/wiki/Brent (to help recognize when you or someone on your team is becoming one)
  163. ↑ Sad alternative outcome from The Phoenix Project for what happens when a "Brent" is not managed correctly: https://medium.freecodecamp.org/we-fired-our-top-talent-best-decision-we-ever-made-4c0a99728fde
  164. ↑ wikipedia: Andon (manufacturing)
  165. ↑ wikipedia: Traditional_lighting_equipment_of_Japan#Andon
  166. ↑ The Andon Cord: https://itrevolution.com/kata/
  167. ↑ Toyota Andon concept at the infamous "MotoMachi factory" in Japan: https://www.youtube.com/watch?v=r_-Pw49ecEU
  168. ↑ Toyota cutting the fabled "andon cord", symbol of Toyota Way: http://www.autonews.com/article/20140805/OEM01/140809892/toyota-cutting-the-fabled-andon-cord-symbol-of-toyota-way (principle not going away, replacing white cords with yellow buttons)
  169. ↑ Incorporating Andon Cord in DevOps: https://www.agiletestingframework.com/incorporating-andon-cord-in-devops/
  170. ↑ The Andon Cord - A Way to Stop Work While Boosting Productivity: https://www.sixsigmadaily.com/what-is-an-andon-cord/
  171. ↑ DevOps, Deming, and how "Pulling the Andon Cord" works in software development: https://caylent.com/devops-deming-andon-cord/
  172. ↑ DevOps Leaders vs. Laggards -- Continuous Testing is a Key Differentiator: https://devops.com/devops-leaders-vs-laggards-continuous-testing-is-a-key-differentiator/
  173. ↑ Agile Vs. DevOps -- 10 Ways They're Different: https://www.informationweek.com/devops/agile-vs-devops-10-ways-theyre-different/d/d-id/1326121
  174. ↑ Waterfall to Agile to DevOps -- The State of Stagnant Evolution: https://devops.com/waterfall-agile-devops-state-stagnant-evolution/
  175. ↑ DevOps -- Who Does What (Part 1): https://dzone.com/articles/devops-who-does-what
  176. ↑ DevOps on AWS Radio -- Continuous Delivery at Netflix — Adam Jordens (Episode 16) (PODCAST): https://dzone.com/articles/devops-on-aws-radio-continuous-delivery-at-netflix
  177. ↑ DevOps Case Study -- Netflix and the Chaos Monkey: https://insights.sei.cmu.edu/devops/2015/04/devops-case-study-netflix-and-the-chaos-monkey.html
  178. ↑ All-day DevOps conference 2017: https://www.alldaydevops.com/2017-all-day-devops-video-recordings (all speakers)
  179. ↑ AWS Public Sector Summit 2016 -- DevOps track - Getting out of Operations Hell: https://www.slideshare.net/AmazonWebServices/devops-tech-talk-getting-out-of-operations-hell-aws-public-sector-summit-2016
  180. ↑ Amazon AWS - DevOps model defined: https://aws.amazon.com/devops/what-is-devops/
  181. ↑ How Amazon handles a new software deployment every second: https://www.zdnet.com/article/how-amazon-handles-a-new-software-deployment-every-second/
  182. ↑ Amazon deploys every 11.6 seconds (2011): https://news.ycombinator.com/item?id=2971521
  183. ↑ Velocity 2011 -- Jon Jenkins, "Velocity Culture": https://www.youtube.com/watch?v=dxk8b9rSKOo | SLIDES
  184. ↑ The Story of Apollo - Amazon’s Deployment Engine: https://www.allthingsdistributed.com/2014/11/apollo-amazon-deployment-engine.html
  185. ↑ AWS re:Invent 2015 -- DevOps at Amazon - A Look at Our Tools and Processes (DVO202): https://www.youtube.com/watch?v=esEFaY0FDKc | SLIDES
  186. ↑ Why the DevOps faithful keep pulling away from their competitors: https://www.techrepublic.com/article/why-the-devops-faithful-keep-pulling-away-from-their-competitors/
  187. ↑ 10 companies killing it at DevOps: https://techbeacon.com/10-companies-killing-it-devops
  188. ↑ Adobe turns to DevOps to manage cloud deliveries: http://blogs.wsj.com/cio/2015/06/15/adobe-turns-to-devops-platform-to-manage-cloud-delivery/
  189. ↑ Netflix earnings, DevOps & profitability: https://www.onpage.com/netflix-earnings-devops-profitability/
  190. ↑ What can you do when the pup of programming becomes the black dog of burnout? Dude, leave: https://www.theregister.co.uk/2018/06/18/burn_out_and_leave_patching_the_patcher/
  191. ↑ DevOps - working to prevent "Karōsu" & "Karōjisatsu" (aka. "death by work" & "burnout"): https://itrevolution.com/karojisatsu/
  192. ↑ Google .vs. Amazon's approaches to DevOps: https://plus.google.com/+RipRowan/posts/eVeouesvaVX
  193. ↑ Is DevOps Bullshit?: https://www.linkedin.com/pulse/devops-bullshit-joachim-bauernberger/
  194. ↑ Twitter feed -- DevOps Borat: https://twitter.com/DEVOPS_BORAT
  195. ↑ Continuous Delusion at the Infrastructure Layer: http://cloudscaling.com/blog/devops/continuous-delusion-at-the-infrastructure-layer/
  196. ↑ The Five Ideals and The Unicorn Project: https://dzone.com/articles/announcing-the-unicorn-project-availability-of-fir
  197. ↑ Going to Market Faster -- Most Companies Are Deploying Code Weekly, Daily, or Hourly: https://blog.newrelic.com/technology/data-culture-survey-results-faster-deployment/
  198. ↑ DevOps & Security - Chaos Monkeys: https://blog.valbonne-consulting.com/2014/09/30/devops-and-security/
  199. ↑ 2018 State of the Software Supply Chain Report (12 DevSecOps findings): https://www.sonatype.com/hubfs/2018%20State%20of%20the%20Software%20Supply%20Chain%20Report.pdf?t=1538697178154
  200. ↑ Best Practices for Measuring your Delivery Pipeline: https://www.newrelic.com/webinar/best-practices-code-pipeline-180912 | SAMPLE CODE
  201. ↑ The Ultimate DevOps Tools Ecosystem Tutorial, Part I -- Overview - Plan, Develop, Test, Release, Operate: https://dzone.com/articles/the-ultimate-devops-tools-ecosystem-tutorial-part
  202. ↑ The Ultimate DevOps Tools Ecosystem Tutorial, Part II -- Planning: https://dzone.com/articles/the-ultimate-devops-tools-ecosystem-tutorial-part-1
  203. ↑ The Ultimate DevOps Tools Ecosystem Tutorial, Part III -- Developing: https://dzone.com/articles/the-ultimate-devops-tools-ecosystem-tutorial-part-2
  204. ↑ The Ultimate DevOps Tools Ecosystem Tutorial, Part IV -- Testing: https://dzone.com/articles/the-ultimate-devops-tools-ecosystem-tutorial-part-3
  205. ↑ The Ultimate DevOps Tools Ecosystem Tutorial, Part V -- Releasing: https://dzone.com/articles/the-ultimate-devops-tools-ecosystem-tutorial-part-4
  206. ↑ The Ultimate DevOps Tools Ecosystem Tutorial, Part VI -- Operating: https://dzone.com/articles/the-ultimate-devops-tools-ecosystem-tutorial-part-5
  207. ↑ 11 Tools You Must Have in Your DevOps Toolchain (INFOGRAPHIC): https://dzone.com/articles/11-tools-you-must-have-in-your-devops-toolchain
  208. ↑ Complete DevOps (at & with GitLab) is DevOps reimagined. Here's what that looks like: https://about.gitlab.com/2017/10/11/from-dev-to-devops/
  209. ↑ Build Continuous Quality Into Your DevOps Toolchains (GARTNER REPORT): https://www.gartner.com/doc/reprints?id=1-56H26P1&ct=180711&st=sb
  210. ↑ 10 Tips for Enterprise DevOps: https://www.perforce.com/resources/vcs/10-tips-enterprise-devops
  211. ↑ 7 Things Your Boss Doesn’t Get About Software Development: https://dzone.com/articles/7-things-your-boss-doesnt-understand-about-softwar
  212. ↑ Do programmers take advantage of managers that don’t understand code?: https://jaxenter.com/programmers-take-advantage-managers-dont-understand-code-115187.html
  213. ↑ 7 signs you're doing devops wrong: https://www.infoworld.com/article/3011631/devops/7-signs-youre-doing-devops-wrong.html
  214. ↑ DevOps Is Bullshit -- Why One Programmer Doesn’t Do It Anymore: https://lionfacelemonface.wordpress.com/2015/03/08/devops-is-bullshit-why-one-programmer-doesnt-do-it-anymore/
  215. ↑ Why One Programmer Doesn’t "Do DevOps" Anymore (HackerNews discussion): https://news.ycombinator.com/item?id=9170615
  216. ↑ Is DevOps Buillshit (i.e. why it isn't... response to previous blog post): https://blog.valbonne-consulting.com/2015/03/10/is-devops-bullshit/
  217. ↑ Why DevOps (done badly) is burning out developers: https://www.infoworld.com/article/3009004/devops/why-devops-is-burning-out-developers.html
  218. ↑ Doing DevOps? Turn your boss into a DevOps leader first (or suffer): https://devopsagenda.techtarget.com/opinion/Why-DevOps-leaders-are-essential-in-all-levels-of-an-enterprise
  219. ↑ How to convince your boss that it is DevOps that he wants (by Jan de Vries): https://www.slideshare.net/AgileLietuva/jan-de-vries-how-to-convince-your-boss-that-it-is-devops-that-he-wants
  220. ↑ DevOps automation remains a high hurdle for DevOps culture: https://devopsagenda.techtarget.com/opinion/DevOps-automation-remains-the-final-hurdle-for-DevOps-culture
  221. ↑ What DevOps Taught Us in 2018: https://hackernoon.com/what-devops-taught-us-in-2018-e5f6637a4921
  222. ↑ Why you need a DevOps Transformation to Survive: https://www.cloudops.com/2019/01/why-you-need-a-devops-transformation-to-survive/
  223. ↑ 5 Guaranteed Ways to Kill DevOps Developer Productivity: https://devops.com/5-guaranteed-ways-to-kill-devops-developer-productivity/ (Plumbing work, Security after the fact, Status checks, GUI/overhead overload, Tool tyranny)
  224. ↑ Digital Transformation for Modern Enterprises Through DevOps — A Complete Guide: https://hackernoon.com/digital-transformation-for-modern-enterprises-through-devops-a-complete-guide-6f595463c7dd
  225. ↑ How the Flow Framework® Maximizes Your Wins from SAFe®: https://blog.tasktop.com/how-the-flow-framework-maximizes-your-wins-from-safe/
  226. ↑ Signs You’re in a Death Spiral (and How to Turn It around before It’s Too Late): https://www.infoq.com/articles/death-spiral-software/
  227. ↑ Flow Metrics - What They Are & Why You Need Them (SLIDES): https://www.slideshare.net/tasktop/flow-metrics-what-they-are-why-you-need-them-122219343
  228. ↑ Minimize Team Cognitive Load to Increase Flow: https://itrevolution.com/minimize-cognitive-load-of-teams/
  229. ↑ Have you already started to use Infrastructure as Code on your daily work?: https://blogs.sap.com/2014/12/04/have-you-already-started-to-use-infrastructure-as-code-on-your-daily-work/
  230. ↑ IaaS, EaaS, PaaS, and DevOps – Good for Absolutely Everything: https://infocus.dellemc.com/bart_driscoll/iaas-eaas-paas-devops-good-forhuhabsolutely-everything/
  231. ↑ Should We Start Baking Immutable Infrastructure Instead of Frying On-Demand?: https://dzone.com/articles/should-we-start-baking-immutable-infrastructure-in
  232. ↑ Google -- SRE vs. DevOps - competing standards or close friends? (BACKUP): https://cloud.google.com/blog/products/gcp/sre-vs-devops-competing-standards-or-close-friends
  233. ↑ SRE vs DevOps vs Cloud Native (SLIDES): https://www.slideshare.net/rhirschfeld/sre-vs-devops-vs-cloud-native-preso
  234. ↑ Intercom’s Rich Archbold on how to run less software: https://www.intercom.com/blog/podcasts/intercom-rich-archbold-run-less-software/
  235. ↑ Site Reliability Engineering -- How Google Runs Production Systems: https://landing.google.com/sre/sre-book/ (full book by Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Murphy & Google SRE teams)
  236. ↑ NewRelic -- guide to Site Reliability Engineering: https://newrelic.com/resource/site-reliability-engineering
  237. ↑ SREiously -- goal, role & types of SRE: https://www.newrelic.com/webinar/SRE-Principles-180829
  238. ↑ Why Do Great Product Companies Release Software To Production Multiple Times A Day: https://www.netsolutions.com/insights/why-do-great-product-companies-release-software-to-production-multiple-times-a-day/
  239. ↑ 5 things toxic to scalability: https://www.iheavy.com/2011/08/26/5-things-are-toxic-to-scalability/
  240. ↑ SRE vs DevOps: What's the difference?: https://www.cio.com/article/3282424/devops/sre-vs-devops-whats-the-difference.html
  241. ↑ SRE vs. DevOps — a False Distinction?: https://devops.com/sre-vs-devops-false-distinction/
  242. ↑ Google SREs on "What's the Difference Between DevOps and SRE?": https://www.youtube.com/watch?v=uTEL8Ff1Zvk
  243. ↑ SRE vs. DevOps -- Any Common Ground? : https://dzone.com/articles/sre-vs-devops-any-common-ground
  244. ↑ DevOps vs. SRE -- Why not both?: https://blog.netapp.com/blogs/devops-vs-sre-why-not-both
  245. ↑ DevOps and SRE Words Matter - How Our Language has Evolved: https://dzone.com/articles/devops-amp-sre-words-matter-how-our-language-has-e
  246. ↑ The 4 Quality Gates Every SRE Team Must Check Before Promoting Code: https://blog.overops.com/the-4-quality-gates-every-sre-team-must-check-before-promoting-code/
  247. ↑ An awesome roundup of SRE links: https://github.com/dastergon/awesome-sre/blob/master/README.md
  248. ↑ Building a Site Reliability Engineering (SRE) Capability: https://n4stack.io/2018/10/04/sre-managed-service/
  249. ↑ Showing the Value of Site Reliability Engineering (SRE): https://victorops.com/blog/showing-the-value-of-site-reliability-engineering-sre
  250. ↑ Implement Site Reliability Engineering: https://www.ibm.com/garage/method/practices/manage/site-reliability-engineering
  251. ↑ An Infographic for Site Reliability Engineering (SRE): https://medium.com/@yury.nino.roa/an-infographic-for-site-reliability-engineering-sre-cc010c098dea
  252. ↑ Site Reliability Engineering (SRE) overview: http://www.validatek.com/methodologies/site-reliability-engineering-sre
  253. ↑ What SREs Can Learn From Facebook’s Largest Outage: https://dzone.com/articles/what-sres-can-learn-from-facebooks-largest-outage
  254. ↑ Building a Platform and SRE Team: https://www.bigmarker.com/techstrong/Building-a-Platform-and-SRE-Team
  255. ↑ How to Adopt an SRE Practice (When You're not Google): https://devops.com/how-to-adopt-an-sre-practice-when-youre-not-google/
  256. ↑ Value Streams in Software Development: https://www.leanit.co.nz/2014/07/05/value-streams-2/
  257. ↑ Value Stream Mapping is a System Approach…Or Is It?: http://www.leanteamsusa.com/value-stream-mapping-is-a-system-approach/
  258. ↑ Value Stream Mapping to Eliminate Waste in the Supply Chain: https://www.industrystarsolutions.com/blog/2017/11/value-stream-mapping-eliminate-waste/
  259. ↑ Lean meets Industry 4.0 - Value stream thinking as denominator: https://blog.bosch-si.com/industry40/lean-meets-industry-4-0-value-stream-thinking-denominator/
  260. ↑ Value Stream Mapping Guide - Complete VSM Tutorial: https://creately.com/blog/diagrams/value-stream-mapping-guide/
  261. ↑ What is Value Stream Mapping (VSM), Benefits, Process and Value: https://www.plutora.com/blog/value-stream-mapping
  262. ↑ Understanding Value Stream Management (as a software professional): https://www.tasktop.com/value-stream-management
  263. ↑ Values, Leadership, and implementing the Deming philosophy: https://www.youtube.com/watch?v=bu7JMAQggfo
  264. ↑ JFDI — When rationality and common sense leave the building…: https://medium.com/swlh/jfdi-when-rationality-and-common-sense-leave-the-building-4fcad616a129
  265. ↑ US Department of Defense (DoD) -- "Title 10" - Authority & Responsibility: https://risacher.org/jfdi/2013/03/title-10-authority-and-responsibility/
  266. ↑ Netflix -- Migrating to Public Cloud (SLIDES): https://www.slideshare.net/adrianco/migrating-to-public-cloud
  267. ↑ Jedi's Feel Doin' It: https://gist.github.com/v0lkan/2694911 | MANIFESTO (an anti-thesis to command-control JFDI to start JDFI yourself)
  268. ↑ The underlying problem with Congress is deciding how to allocate their time: https://www.vox.com/mischiefs-of-faction/2018/5/8/17328694/congress-time-allocation
  269. ↑ The Tragedy of the Commons: http://godpoliticsbaseball.blogspot.com/2018/01/the-tragedy-of-commons.html
  270. ↑ Tragedy of the Commons and a Land Ethic: https://usu.instructure.com/courses/444419/pages/lecture-4-tragedy-of-the-commons-and-a-land-ethic
  271. ↑ The Tragedy of the Commons - How Elinor Ostrom Solved One of Life’s Greatest Dilemmas: http://blogs.strategygroup.net/wp2/economy/tag/tragedy-of-the-commons/
  272. ↑ Elinor Ostrom's 8 Principles for Managing A Commmons: http://www.onthecommons.org/magazine/elinor-ostroms-8-principles-managing-commmons
  273. ↑ Elinor Ostrom - Nobel Prize Lecture: https://www.nobelprize.org/mediaplayer/?id=1223
  274. ↑ Pluralistic Ignorance and Diffusion of Responsibility: https://neuroinsights.in/2018/04/07/pluralistic-ignorance-and-diffusion-of-responsibility/
  275. ↑ Diffusion of Responsibility -- the Perspective for Bystander Effect from the Chinese Story of Three Monks: https://sites.psu.edu/aspsy/2015/04/06/diffusion-of-responsibility-the-perspective-for-bystander-effect-from-the-chinese-story-of-three-monks/
  276. ↑ Building a culture of Responsibility in CEP Plant Timisoara : https://www.continental-people.com/building-a-culture-of-responsibility-in-cep-plant-timisoara/
  277. ↑ The Bystander Effect - Diffusion of Responsibility: https://passionateinknowledge.com/2017/05/23/the-bystander-effect-diffusion-of-responsibility/
  278. ↑ wikipedia: DIKW pyramid
  279. ↑ wikipedia: Tuckman's stages of group development (Forming, Storming, Norming & Performing)
  280. ↑ MIT - Using the Stages of Team Development: https://hr.mit.edu/learning-topics/teams/articles/stages-development
  281. ↑ Forming, Storming, Norming, and Performing: https://www.mindtools.com/pages/article/newLDR_86.htm
  282. ↑ Question -- What Is The Difference Between Data, Information, Knowledge: https://www.chegg.com/homework-help/questions-and-answers/difference-data-information-knowledge-wisdom-give-specific-examples--might-important-conce-q38191606
  283. ↑ Data, information, knowledge and wisdom: https://www.sciencedirect.com/science/article/abs/pii/S1472029913003056
  284. ↑ The Circle of Data, Information, Knowledge, Understanding, and Wisdom: https://www.taus.net/think-tank/articles/the-circle-of-data-information-knowledge-understanding-and-wisdom
  285. ↑ UNDERSTANDING -- DATA, KNOWLEDGE, INFORMATION & WISDOM: https://kvaes.wordpress.com/2013/05/31/data-knowledge-information-wisdom/
  286. ↑ Strive to get higher on the Data -> Wisdom Continuum — another key tenet: https://medium.com/@Iyer/strive-to-get-higher-on-the-data-information-knowledge-understanding-and-wisdom-continuum-c5ccb96438
  287. ↑ Remarkable Comic Explaining Why It’s Better To Really Understand Few Things Than Partially Know A Lot!: https://www.lifecoachcode.com/2016/09/14/better-really-understand-things-partially-know-lot/
  288. ↑ Good Decision-making (INFOGRAPHIC): https://www.robbieshort.com/design.html
  289. ↑ MITIGATING GROUPTHINK: https://infograph.venngage.com/p/138482/mitigating-groupthink
  290. ↑ Groupthink Kills Collaboration!: https://www.jabra.com/blog/groupthink-kills-collaboration/
  291. ↑ Dotted Line Reporting: https://www.global-integration.com/glossary/dotted-line-reporting/
  292. ↑ What Team Structure is Right for DevOps to Flourish?: https://blog.matthewskelton.net/2013/10/22/what-team-structure-is-right-for-devops-to-flourish/
  293. ↑ The State of DevOps 2019 -- A Roundup of “State of DevOps” Reports: https://www.bmc.com/blogs/state-of-devops/
  294. ↑ 2019 Accelerate State of DevOps Report: https://cloud.google.com/devops/state-of-devops/

See Also

Agile | IT | Productivity | CI/CD | Automated Testing | Optimization/Metrics | Analytics | Dashboard

Retrieved from "http://wiki.bcmoney-mobiletv.com/index.php?title=DevOps&oldid=60435#Metrics"