Optimization

From BC$ MobileTV Wiki
(Redirected from KPM)
Jump to: navigation, search

It's important to optimize any application before launching it on a large scale, but this is particularly true of modern Mobile Applications and Web Applications, as they may potentially experience viral (also known as "web-scale") growth. Characteristics of these applications is "bursty traffic" and "sudden bandwidth needs" as at particular times of day, week, month or year users flock to the site; while other times it may sit idle or sparsely used. As a result there is a high need to rapidly scale modern applications to peak demand, and where practical/possible, be able to scale capacity back down in order to save on costs. The perfect optimization pattern is thus one which can support the highest possible demands but cost the least amount of money by only scaling up when necessary, and scaling down resources in all other occasions.


Specifications


Metrics

Organizational Performance

For more organizational performance metrics, see: DevOps metrics

Website Performance

For more on how Search Engines like google measure and weight "website performance" metrics, see Web Vitals


[13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33]


OGSM

[34]

OKR

[35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55]

KRA

Key Result Area (KRA) is a metric representative of individual performance towards an organizational goal.

KRI

[57]

KPI

Key Performance Indicators (KPIs) are types of events, transaction, or simply actions taken upon a thing, which are theorized to be beneficial to measure. The purpose of the measurement is most useful when it is representative of progress towards achieving a specific goal.

The following are the most commonly used general KPIs across all sizes and types of companies:

  • Bounce rate
  • Page view time (aka. time-on-page, but also worth measuring cumulative time-on-site)
  • Content view time (time spent on specific or cumulative on all content within a page)
  • Content completions (i.e. video/audio/ad complete views)
  • Conversion rate
  • Purchase rate
  • Subscription rate
  • Monthly Recurring Revenue (MRR)
  • Cost of Goods Sold (COGS)
  • Average Revenue Per Unit (ARPU)
  • Cost Per Acquisition (CPA; single event)
  • Customer Acquisition Cost (CAC; repeat events)
  • Customer Life-Time Value (LTV)
  • Customer attrition/loss rate (Churn)
  • Compound Annual Growth Rate (CAGR)

Some E-Commerce specific metrics include

  • Page Load time
  • API request/response time:
  • Average order value
  • Products added to cart
  • Products ordered (depth/breadth)
  • Total Revenue per Product line
  • Products Browsed
  • Total pages visited
  • Frequency of Page visits
  • Total visits by all users (per page)
  • Click path
  • Purchase path (also known as "conversion funnel")

[58] [59] [60] [61]

Advertising metrics:

  • Return On Advertising Spend (ROAS)

[62] [63] [64] [65]

Social Media has its own set of interaction-focused metrics:

  • View count
  • Open rate (by target segment)
  • Share rate
  • Comment count
  • Like/vote count (and relates "+/-" trends)
  • Subscribe rate
  • Notification opt-in rate


[66] [67] [68] [69] [70] [71] [72] [73] [74] [75] [76] [77] [78] [79] [80] [81] [82] [83] [84] [85] [86] [87] [88] [89] [90] [91] [92]


KPP

Key Performance Parameters (KPP) are the individual aspects of a KPI that can be experimented with to measure their impact on the KPI.


KPM

KPM is the measurement of a KPI (i.e. # of an event, % ROI or rates, $ amount, etc).

[93] [94] [95] [96] [97] [98]

SLA

Service Level Agreement (SLA). [99] [100]

SLO

Service Level Objective (SLO). [101] [102] [103]

SLI

Service Level Indicator (SLI). [104]

Monitoring

A large part of software or hardware optimization is monitoring, where performance is analyzed over time, and errors are logged, flagged and shared with team members, ideally in real-time.

  • See also: Logging (Log Monitoring)
  • See also: Security (Security Monitoring)

[105]

Network

System

Website

BAM

Business activity monitoring (BAM)

BTM

Business transaction management (BTM)

Disaster recovery

[106]

Performance

Performance is broadly the capabilities of a system to meet the demands of end-users. It can be tested, reported on, and measured over time, via various types of Performance Testing. [107] [108] [109] [110] [111] [112] [113] [114] [115] [116] [117] [118]

Profiler

[119] [120]

JMX

[129] [130]

APM

Application Performance Management (commonly abbreviated APM, wherein the "M" could also refer to "Monitoring") is the monitoring and/or management of performance and availability of software applications. APM strives to detect and diagnose complex application performance problems to maintain an expected and minimally acceptable level of service. APM is "the translation of IT metrics into business value" in this sense, with aims of minimizing errors, minimizing loading times, managing resource usage (CPU, memory, storage, etc) and maximizing efficiency as it pertains to the monitored applications in order to deliver business products and services with a high degree of availability, reliability and without technical impediments to customers. APM can be accomplished through Performance Testing, real-time server monitoring, and constant database/application tuning.

[137] [138] [139]


Log Analysis

[140]


Splunk

Splunk is a leading Log Analyzer tool.

[145]


OverOps

OverOps (originally Takipi) is a comprehensive solution that offers:

  1. Log Analysis
  2. Debugging
  3. Profiling
  4. Line-of-code Error detection/traceability
  5. Pre-emptive problem notications (so your team finds out about bugs before customers)

OverOps adds a link to every error in your logs directly to its cause and enables you to see DEBUG log statements and call stacks for any log error directly from the JVM, even if they were not written to the log file.

[146] [147] [148] [149] [150] [151] [152] [153] [154] [155]


Observability

[156] [157] [158] [159] [160]

[161]

Telemetry

[162]

OpenTracing

[163] [164]

OpenTelemetry

[165] [166] [167]

OpenCensus
  • OpenCensus: https://opencensus.io (collect telemetry like metrics & distributed traces from your services)

Approaches

Business Performance

DBR

Drum-Buffer-Rope (commonly abbreviated DBR) is a manufacturing execution methodology based on the fact the output of a system can only be the same as the output at the constraint of the system. Any attempt to produce more than what the constraint can process just leads to excess inventory piling up.[citation needed] The method is named for its three components. The drum is the physical constraint of the plant: the work center or machine or operation that limits the ability of the entire system to produce more. The rest of the plant follows the beat of the drum. Schedule at the drum decides what the system should produce, in what sequence to produce and how much to produce. They make sure the drum has work and that anything the drum has processed does not get wasted.

The buffer protects the drum, so that it always has work flowing to it. Buffers in DBR provide the additional lead time beyond the required set up and process times, for materials in the product flow. Since these buffers have time as their unit of measure, rather than quantity of material, this makes the priority system operate strictly based on the time an order is expected to be at the drum. Each work order will have a remaining buffer status that can be calculated. Based on this buffer status work orders can be color coded into Red, Yellow and Green. The red orders have the highest priority and must worked on first since they have penetrated most into their buffers followed by yellow and green. As time evolves this buffer status might change and the color assigned to the particular work order change with it.


Dashboard

A Dashboard is a visual, informational resource used to help inform decision making.

[168]

Back-end Optimization

Memory Management


Distributed Computing


Thread Management

[169] [170] [171] [172] [173] [174] [175] [176] [177] [178] [179] [180] [181]

Heap Dump Analysis

[182] [183] [184] [185] [186]


Front-end Optimization

[187] [188] [189] [190] [191] [192]


3rd Party JavaScript


Lazy-loading

content-visibility

[219] [220] [221] [222]

Event Handling


Scroll Listeners


External Website Links

Fullstack Optimization

YSlow performance criteria

YSlow was a long-time industry leader in performance optimization for the web, along with YUI compressor for CSS/JS minification. Since it was tailored to HTTP 1.1/1.2 specs, it is slowly getting replaced by other tools more optimized for HTTP/2.0 as the once stacked, but now fledgling Yahoo! development team struggles to support legacy tools and open source projects (sunsetting most since their acquisition by Verizon[237]).

  1. Minimize HTTP Requests
  2. Use a Content Delivery Network
  3. Avoid empty src or href
  4. Add an Expires or a Cache-Control Header
  5. Gzip Components
  6. Put StyleSheets at the Top
  7. Put Scripts at the Bottom
  8. Avoid CSS Expressions
  9. Make JavaScript and CSS External
  10. Reduce DNS Lookups
  11. Minify JavaScript and CSS
  12. Avoid Redirects
  13. Remove Duplicate Scripts
  14. Configure ETags
  15. Make AJAX Cacheable
  16. Use GET for AJAX Requests
  17. Reduce the Number of DOM Elements
  18. No 404s
  19. Reduce Cookie Size
  20. Use Cookie-Free Domains for Components
  21. Avoid Filters
  22. Do Not Scale Images in HTML
  23. Make favicon.ico Small and Cacheable

[238] [239] [240] [241] [242] [243] [244] [245] [246]

Tools

[250] [251] [252] [253] [254]

[255]


Resources


Tutorials

[271] [272] [273] [274]


External Links


References

  1. wikipedia: Apdex
  2. JMeter Beginner tutorial -- Generating Report Dashboard: https://www.youtube.com/watch?v=aHV5IizW7u4
  3. What are MTTR and MTTI?: https://www.sumologic.com/mttr-mtti/
  4. Google dev docs -- Web fundamentals - Critical Rendering Path: https://developers.google.com/web/fundamentals/performance/critical-rendering-path/
  5. MozDev -- Web Performance - Critical rendering path: https://developer.mozilla.org/en-US/docs/Web/Performance/Critical_rendering_path
  6. Understanding the Critical Rendering Path: https://bitsofco.de/understanding-the-critical-rendering-path/
  7. How to Optimize the Critical Rendering Path in WordPress: https://kinsta.com/blog/critical-rendering-path/
  8. At Google I/O 2016, we launched the brand-new Polymer App Toolbox for building Progressive Web Apps: https://www.polymer-project.org/blog/2016-05-19-Polymer-IO-2016
  9. Faster Web App Delivery with PRPL: https://addyosmani.com/blog/the-prpl-pattern/
  10. PRPL pattern (with examples) - Solutions for modern web app optimization: https://blog.logrocket.com/prpl-pattern-solutions-for-modern-web-app-optimization/
  11. Polymer (which pioneered the concept, explains how it handles) PRPL Pattern: https://polymer-library.polymer-project.org/2.0/docs/apps/prpl
  12. Google dev docs -- Apply instant loading with the PRPL pattern: https://web.dev/apply-instant-loading-with-prpl/
  13. Understanding Agile Team Metrics — Measure Many Things: http://www.andycleff.com/2019/09/understanding-agile-team-metrics/
  14. THE IMPACT OF AGILE QUANTIFIED -- SWAPPING INTUITION FOR INSIGHTS: http://maccherone.com/share/Impact of Agile Quantified - 2013 edition.pdf
  15. Quantifying the Impact Of Agile Practices: hjttp://maccherone.com/publications/KEYNOTE-MetricsAndInsights-Maccherone-0604-1245PM.pdf
  16. What to Measure and Why - Kanban Metrics: https://getnave.com/blog/kanban-metrics/
  17. Outcome, Decision, Insight, Measure (ODIM) - A framework for choosing the right metrics: http://maccherone.com/larry/2017/09/13/odim-a-framework-for-choosing-the-right-metrics/
  18. Three Secrets to Successful Agile Metrics: https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=503751
  19. The secret to DevOps metrics? It's easy -- just use them: http://searchsoftwarequality.techtarget.com/feature/The-secret-to-DevOps-metrics-Its-easy-just-use-them
  20. DevOps metrics matter -- How to prove business value: https://techbeacon.com/devops-metrics-matter-how-prove-business-value
  21. DevOps – Metrics, Goals and Waste: https://devopsdayslondon2016.wordpress.com/2015/11/20/devops-metrics-goals-and-waste/
  22. 8 Ways to Track the Success of DevOps Teams: https://dzone.com/articles/8-ways-to-track-the-success-of-your-devops-team
  23. 4 Balanced Metrics for Tracking Agile Teams: https://www.agileconnection.com/article/4-balanced-metrics-tracking-agile-teams
  24. Five agile metrics you won't hate: https://www.atlassian.com/agile/project-management/metrics
  25. From a vision to metrics: https://www.jamsovaluesmarter.com/metrics
  26. Be Careful What You Measure - 10 Metrics For Agile Teams: https://platinumedge.com/blog/be-careful-what-you-measure-10-metrics-agile-teams
  27. Stop Measuring: https://medium.com/@jardenalondon/stop-measuring-468f99c7078
  28. Eric Ries on the Ineffectiveness of Vanity Metrics: https://www.youtube.com/watch?v=0dvsNmL__9U
  29. "Agile metrics" are the key to your team’s continuous improvement: https://www.sumoheavy.com/agile-metrics-are-the-key-to-your-teams-continuous-improvement/
  30. 60 Important Business Metrics Your Company Must Know: https://dzone.com/articles/60-important-business-metrics-your-company-must-kn
  31. A Guide to Product Manager KPIs: https://www.productplan.com/learn/product-manager-kpis/
  32. Measure Software Delivery Performance with Four Key Metrics: https://itrevolution.com/measure-software-delivery-performance-four-key-metrics/
  33. Metrics to track and drive your DevOps maturity: https://sdtimes.com/wp-content/uploads/2021/04/Plandek-Whitepaper-Metrics-to-track-and-drive-Agile-DevOps-maturity.pdf?email=bryan.copeland%40alc.ca&JOBFUNCTION=Software%20Engineer&INDUSTRY=Media%2F%20Marketing%2F%20Entertainment%2F%20Travel%2F%20Hospitality&NUMBEROFEMPLOYEES=100-249%20 (from "beginner" to "advanced")
  34. Leveraging Mobile Infrastructure with Data-Driven Decisions: https://engineering.atspotify.com/2020/07/22/leveraging-mobile-infrastructure-with-data-driven-decisions/
  35. The Ultimate Guide to OKR: https://www.perdoo.com/okr/
  36. All About OKRs -- How To Set Them, Achieve Them, And Track Them In Trello: http://blog.trello.com/okrs-set-achieve-track-trello
  37. Atlassian's "Publicly announced" Marketing Team OKRs for 2019-Q2: https://paper.dropbox.com/doc/Public-Marketing-Team-OKRs-Q2-kK7a5lgTf13A0R8cg5KRE
  38. Weekdone's OKR Goal setup template: https://weekdone.com/ebook/okr-goal-setting-guide-template/
  39. The Management Framework that Propelled LinkedIn to a $20 Billion Company: http://firstround.com/review/the-management-framework-that-propelled-LinkedIn-to-a-20-billion-company/
  40. Startup Lab workshop -- How Google sets goals - OKRs: https://www.youtube.com/watch?v=mJB83EZtAjc
  41. What's the difference between KPIs (key performance indicators) and OKRs (objective and key results)?: https://www.quora.com/Whats-the-difference-between-KPIs-key-performance-indicators-and-OKRs-objective-and-key-results
  42. 5 Steps to Setting and Achieving Business Goals in 2019: https://blog.weekdone.com/5-steps-to-setting-and-achieving-business-goals-in-2019/
  43. 29 "Performance Appraisal" methods explained: https://grosum.com/blog/performance-appraisal-methods/
  44. Objectives and Key Results (OKR) -- How to implement the #1 goal setting framework in Asana: https://forum.asana.com/t/objectives-and-key-results-okr-how-to-implement-the-1-goal-setting-framework-in-asana/4539
  45. OKRs vs. SMART Goals: https://www.perdoo.com/blog/okr-vs-smart-goals/
  46. What is the difference between MBO and OKR: https://upraise.io/blog/difference-mbo-okr/
  47. Creating GREAT OKRs and a great quarterly planning process: https://www.slideshare.net/7Geese/creating-great-okrs-and-a-great-quarterly-planning-process
  48. Goal setting is important for you and business: https://www.jamsovaluesmarter.com/goal-setting/
  49. OKRs - goal setting that focuses on outcomes: https://www.atlassian.com/agile/agile-at-scale/okr
  50. The North Star Playbook (PDF): https://info.amplitude.com/rs/138-CDN-550/images/Amplitude-The-North-Star-Playbook.pdf
  51. OKRs & DevOps - From Micromanagement Misery to Finding Flow: https://itrevolution.com/okrs-from-micromanagement-misery-to-finding-flow/
  52. OKRs Examples for Software Companies: https://blog.tasktop.com/okrs-examples-for-software-companies/
  53. Use OKRs to Set Goals for Teams, Not Individuals: https://hbr.org/2020/12/use-okrs-to-set-goals-for-teams-not-individuals
  54. How Technology Teams Can Thrive with Flow-Based OKRs (E-BOOK): https://go.tasktop.com/how-technology-teams-can-thrive-with-flow-based-okrs.html | WEBINAR
  55. OKR Examples for Software Companies: https://blog.tasktop.com/okrs-examples-for-software-companies/
  56. Definition of KPAs: https://economictimes.indiatimes.com/definition/key-result-areas
  57. How to measure the accuracy of forecasts: https://blog.asmartbear.com/forecast.html
  58. Google research on Page Load Time reduction to Conversion Rate improvements: https://www.thinkwithgoogle.com/marketing-strategies/app-and-mobile/mobile-page-speed-new-industry-benchmarks-load-time-vs-bounce/
  59. Deloitte Digital - Milliseconds make Millions: https://www2.deloitte.com/content/dam/Deloitte/ie/Documents/Consulting/Milliseconds_Make_Millions_report.pdf (study on how improvements in mobile site speed positively affect a brand’s bottom line)
  60. 12 Statistics That Show How Page Load Time Impacts Conversion Rate: https://blog.hubspot.com/marketing/page-load-time-conversion-rates
  61. Wayward KPIs -- Marketers Are Unsure Whether They Accurately Track Them: https://www.mediapost.com/publications/article/372967/wayward-kpis-marketers-are-unsure-whether-they-ac.html
  62. Using ROAS to Measure Advertising Results: https://www.practicalecommerce.com/Using-ROAS-to-Measure-Advertising-Results
  63. Guide to Understanding "Return on Ad Spend" (ROAS): https://www.wallstreetprep.com/knowledge/return-on-ad-spend-roas/
  64. Vendor Spend Analysis - How and Why You Should Be Monitoring: https://www.venminder.com/blog/vendor-spend-analysis-monitoring
  65. The Plain English Guide to Return on Ad Spend (ROAS): https://blog.hubspot.com/marketing/return-on-ad-spend
  66. SaaS Metrics – A Guide to Measuring and Improving What Matters: http://www.forentrepreneurs.com/saas-metrics/
  67. SaaS Metrics 2.0 – A Guide to Measuring and Improving what Matters: https://www.forentrepreneurs.com/saas-metrics-2/
  68. How to Find Key Performance Indicators (KPIs) with Examples: https://www.optimizesmart.com/understanding-key-performance-indicators-kpis-just-like-that/
  69. How Loading Time Affects Your Bottom Line: https://neilpatel.com/blog/loading-time/
  70. What is CAGR? Making Sense of a Commonly Used Metric: https://www.asimplemodel.com/reference/63/what-is-cagr-making-sense-of-a-commonly-used-metric/
  71. Understand your Online Gambling Business with Key Performance Indicators: https://everymatrix.com/blog/gambling-business-key-performance-indicators.html
  72. 18 Key Performance Indicator (KPI) examples defined: https://www.clearpointstrategy.com/18-key-performance-indicators/
  73. The 10 Marketing KPIs You Should Be Tracking: https://www.impactbnd.com/the-10-marketing-kpis-you-should-be-tracking
  74. KPI templates & examples: https://www.accelerate-agency.com/kpi-key-performance-indicator-examples-templates
  75. How to Manage, Visualize KPIs for Ecommerce: https://www.practicalecommerce.com/how-to-identify-visualize-kpis-for-ecommerce
  76. KPI guide for GooglePlay Apps & Games -- Part 1 of 4 - Intro to (Android app) Metrics: https://medium.com/googleplaydev/a-kpis-guide-for-google-play-apps-and-games-intro-to-metrics-214da979f5a2 | VIDEO
  77. 14 Important KPIs To Help You Track Your Digital Transformation: https://www.forbes.com/sites/forbestechcouncil/2020/06/25/14-important-kpis-to-help-you-track-your-digital-transformation/
  78. Measuring Usability With The System Usability Scale (SUS): https://measuringu.com/sus/
  79. Metrics That Matter -- PPC: https://www.axis41.com/blog/sem/metrics-that-matter-ppc/
  80. Metrics That Matter -- Email Marketing: https://www.axis41.com/blog/analytics/metrics-that-matter-email-marketing/
  81. Metrics That Matter -- Social Media Marketing: https://www.axis41.com/blog/marketing-strategy/metrics-that-matter-social-media-marketing/
  82. 21 Most Important Marketing Effectiveness Metrics You Should Know: https://ninepeaksmedia.com/21-most-important-marketing-effectiveness-metrics-you-should-know
  83. Why LTV/CAC Ratio is the Lifeblood of SaaS + How to Calculate It: https://www.profitwell.com/recur/all/youre-calculating-cac-wrong
  84. What is the "LTV:CAC" ratio?: https://www.geckoboard.com/best-practice/kpi-examples/ltv-cac-ratio/
  85. How to Calculate the LTV:CAC Ratio for Your SaaS Business: https://medium.com/corl/how-to-calculate-lifetime-value-customer-acquisition-cost-ratio-for-your-saas-business-332510188726
  86. What Does Your LTV/CAC Ratio Tell You?: https://www.lightercapital.com/blog/what-does-ltv-cac-ratio-tell-you/
  87. LTV:CAC = Customer Lifetime Value / Customer Acquisition Cost (example): https://www.klipfolio.com/resources/kpi-examples/saas/customer-lifetime-value-to-customer-acquisition-cost
  88. Key Performance Indicators (INFOGRAPHIC): https://www.kpiinstitute.org/key-performance-indicators-infographic/ | SRC
  89. Key Performance Indicators (INFOGRAPHIC): https://issuu.com/thekpiinstitute/docs/the-kpi-institute-kpi-training-info
  90. 25 KPIs from Manufacturing (INFOGRAPHIC): https://www.rhythmsystems.com/hs-fs/hubfs/16_RS_For_Blogs/25 KPI Example in Man.png?width=600&name=25 KPI Example in Man.png
  91. 7 Top Notch Marketing KPI examples: https://brandongaille.com/7-top-notch-marketing-kpi-examples/
  92. Favorite E-Commerce KPIs: https://www.practicalecommerce.com/my-favorite-ecommerce-kpis
  93. KPIs vs Metrics: https://www.e-nor.com/blog/general/kpis-vs-metrics
  94. KPIs vs. Metrics in Marketing (INFOGRAPHIC): http://www.thedata.co/kpis-vs-metrics/
  95. Business metrics vs KPIs. What’s the difference?: https://www.klipfolio.com/blog/business-metrics-vs-kpis
  96. Difference Between Metrics and KPIs: https://www.differencebetween.com/difference-between-metrics-and-vs-kpis/
  97. A Metric Leading to Agility: https://ronjeffries.com/xprog/articles/jatrtsmetric/
  98. Metrics Based Process Mapping (MBPM): https://www.redhat.com/en/blog/metrics-based-process-mapping-mbpm
  99. 50 Service Level Agreement (SLA) resources: https://blog.catchpoint.com/2020/01/08/service-level-agreements-resources/
  100. 99.999% uptime vs 99.9% uptime -- the difference two extra “nines” makes: https://www.intermedia.net/blog/2014/01/28/99-999-uptime-vs-99-9-uptime-the-difference-two-extra-nines-makes/ (everyone's goal of "better than four nines", i.e. "five nines and up" and what that really means)
  101. The Calculus of Service Availability: https://queue.acm.org/detail.cfm?id=3096459
  102. A Service-Level What?: https://tanzu.vmware.com/content/blog/a-service-level-what (SLA, SLO & SLI in more detail)
  103. What Are Service-Level Objectives? Lessons Learned: https://www.blameless.com/blog/service-level-objectives-slos-lessons-learned
  104. How to Create Programmatic Service Level Indicators (SLIs): https://blog.newrelic.com/engineering/programmatic-service-level-indicator/
  105. Apache Kafka in the Gaming Industry — Use Cases + Architectures: https://dzone.com/articles/apache-kafka-in-the-gaming-industry-use-cases-arch (event streaming real-world examples such as: Real-time Telemetry, Payments/Bets engine, Ad display & In-App purchase transactions, Monitor data events, Analytics on Marketing campaigns)
  106. Error handling in AWS AppSync: https://advancedweb.hu/error-handling-in-appsync/
  107. wikipedia: Performance Rating
  108. wikipedia: Computer performance
  109. wikipedia: Network performance
  110. wikipedia: Performance engineering
  111. wikipedia: Software performance testing
  112. wikipedia: Performance measurement
  113. wikipedia: Performance tuning
  114. wikipedia: Performance management
  115. wikipedia: Performance improvement
  116. Your 2017 Guide to Retail Performance Success: https://www.soasta.com/your-2017-guide-to-retail-performance-success/
  117. Cost of Performance problems (INFOGRAPHIC): https://www.slideshare.net/mobile/appdynamics/infographic-the-cost-of-performance-issues
  118. Six Dimensions of Performance: https://www.flightlevels.io/blog/six-dimensions-of-performance/
  119. Profiling With JVisualVM : https://dzone.com/articles/profiling-with-jvisualvm
  120. Java performance profiling using flame graphs: https://medium.com/@maheshsenni/java-performance-profiling-using-flame-graphs-e29238130375
  121. jConsole runtime args: http://docs.oracle.com/javase/7/docs/technotes/tools/share/jconsole.html
  122. Monitoring and Management Using JMX -- Enabling the JMX Agent: http://docs.oracle.com/javase/1.5.0/docs/guide/management/agent.html#jmxagent
  123. Using JConsole to Monitor Applications: http://www.oracle.com/technetwork/articles/java/jconsole-1564139.html
  124. Using jConsole as per Java 1.7: http://docs.oracle.com/javase/7/docs/technotes/guides/management/jconsole.html
  125. Using jConsole as per Java 1.5: http://www.cs.mun.ca/java-api-1.5/guide/management/jconsole.html
  126. Standard MBeans (JMX) example: http://docs.oracle.com/javase/tutorial/jmx/mbeans/standard.html
  127. How to activate JMX on my JVM for access with jconsole?: https://stackoverflow.com/questions/856881/how-to-activate-jmx-on-my-jvm-for-access-with-jconsole
  128. Has anyone ever got a remote JMX JConsole to work?: https://stackoverflow.com/questions/151238/has-anyone-ever-got-a-remote-jmx-jconsole-to-work
  129. Profiling the profilers: http://www.javaworld.com/article/2073674/build-ci-sdlc/profiling-the-profilers.html
  130. A Step By Step Guide to Tomcat Performance Monitoring: https://stackify.com/tomcat-performance-monitoring/
  131. Alternatives to AppDynamics: http://alternativeto.net/software/appdynamics/
  132. Web Browser Automation Tools: https://www.monitis.com/blog/web-browser-automation-tools/
  133. Monitor Everything with Monitis – And do it easily with PowerShell (13-part series): http://www.monitis.com/blog/monitor-everything-with-monitis-and-do-it-easily-with-powershell-part-13/ (see parts 4 & 5)
  134. What are some good cloud-based website monitoring tools?: http://www.quora.com/What-are-some-good-cloud-based-website-monitoring-tools
  135. Monitis transaction – a rapid implementation web-monitoring solution (based on Selenium): http://codevoyagers.com/2016/04/14/monitis-transaction-a-rapid-implementation-web-monitoring-solution/
  136. wikipedia: Xymon
  137. Gartner -- Magic Quadrant (2021) - Application Performance Monitoring: https://www.gartner.com/doc/reprints?id=1-25S66H1I&ct=210412&st=sb&mkt_tok=MDMxLVdJWC02MTgAAAF8cAkZc7XA-xFYtggoMtdpCuCpsxSRmBwb0tqz8ZXbkUd6sE27ZDKRc_dd6sndEj3fiaz8riTu9d0h5lSvSyEU7CTIlov79l-nI0qK8x8GWLKcYQ
  138. Gartner - Innovation Insight for Observability: https://www.gartner.com/doc/reprints?id=1-24C5R1YO&ct=201008&st=sbf
  139. Observability won’t replace monitoring (because it shouldn’t): https://go.lightstep.com/rs/260-KGM-472/images/observability-replace-monitoring-guide.pdf
  140. Understanding and Leveraging the Java Stack Trace: https://stackify.com/java-stack-trace/
  141. Splunk Community Projects - Splunk Java Logging Framework: http://dev.splunk.com/view/community-projects/SP-CAAAED7
  142. Introduction to Splunk HTTP Event Collector: http://dev.splunk.com/view/event-collector/SP-CAAAE6M
  143. Building in telemetry with high-performance data collection: http://dev.splunk.com/view/dev-guide/SP-CAAAE7B
  144. How to log to HTTP Event Collector using Splunk logging for JavaScript: http://dev.splunk.com/view/splunk-logging-javascript/SP-CAAAE64
  145. JMeter output to Splunk event log: https://stackoverflow.com/questions/12704972/jmeter-output-to-event-log (best solution is to use a "Beanshell Post-Processor" to collect the data and send via TCP directly into Splunk)
  146. AppDynamics vs Dynatrace: Battle of the Enterprise Monitoring Giants: http://blog.takipi.com/appdynamics-vs-dynatrace-battle-of-the-enterprise-monitoring-giants/
  147. AppDynamics VS New Relic - Which Tool is Right For You? The Complete Guide: http://blog.takipi.com/appdynamics-vs-new-relic-which-tool-is-right-for-you-the-complete-guide/
  148. New Relic vs. AppDynamics: http://www.myloadtest.com/new-relic-vs-appdynamics/
  149. CA - APM 10: http://www3.ca.com/us/products/application-performance-management.aspx | BOOK (formerly WilyTech, acquired by CA)
  150. wikipedia: CA Technologies (known more commonly by former name "Computer Associates International, Inc.", they are a major M&A tech holding company)
  151. The Wily Introscope Profiler: https://docs.oracle.com/cd/E19501-01/819-3659/beafq/index.html
  152. Magic Quadrant for Application Performance Monitoring Suites: https://www.gartner.com/doc/reprints?id=1-3OIV5U7&ct=161222&st=sb
  153. Are there any good open source APM (Application Performance Management) products available?: http://www.quora.com/Are-there-any-good-open-source-APM-Application-Performance-Management-products-available
  154. What are the top Application Performance Management (APM) companies/services?: http://www.quora.com/What-are-the-top-Application-Performance-Management-APM-companies-services
  155. The Top 5 Disadvantages of Not Implementing an Exception Inbox Zero Policy: http://blog.takipi.com/the-top-5-disadvantages-of-not-implementing-an-exception-inbox-zero-policy/
  156. IBM - What is "Observability"?: https://www.ibm.com/cloud/learn/observability
  157. Splunk - What is "Observability"?: https://www.splunk.com/en_us/data-insider/what-is-observability.html
  158. Application Performance Monitoring/Management (APM) resources: https://www.alexanderpodelko.com/PerfManagement.html#PerformanceManagement
  159. Introduction to Observability: https://docs.honeycomb.io/learning-about-observability/intro-to-observability/
  160. Observability in 2020 -- A Manifesto: https://newrelic.com/resources/ebooks/observability-2020-manifesto
  161. State of Observability 2022 -- Modernization Cannot Succeed without Observability: https://tanzu.vmware.com/content/blog/state-of-observability-2022-modernization
  162. Understanding Istio Telemetry v2: https://blog.christianposta.com/understanding-istio-telemetry-v2/
  163. OpenTracing on Kubernetes — Trace your existing java apps without touching a line of code: https://medium.com/opentracing/opentracing-on-kubernetes-get-yer-tracing-for-free-7a69cca03c8a
  164. Spring Boot App Distributed Tracing on steroids with OpenJDK Flight Recorder (and OpenTracing): https://dzone.com/articles/how-to-accelerate-distributed-tracing-for-spring-b
  165. OpenTelemetry docs -- What is OpenTelemetry?: https://opentelemetry.lightstep.com/
  166. NewRelic - What is OpenTelemetry and why is it the future of instrumentation?: https://newrelic.com/blog/best-practices/what-is-opentelemetryzh
  167. Jenkins plugins -- OpenTelemetry: https://plugins.jenkins.io/opentelemetry/
  168. Dashboards and Culture - How Openness Changes Your Behavior: https://www.infoq.com/presentations/dashboard-culture-ibm/
  169. wikipedia: Thread (computing)
  170. wikipedia: Thread safety
  171. wikipedia: Java concurrency
  172. Java Thread Tutorial -- Creating Threads and Multithreading in Java: https://dzone.com/articles/java-thread-tutorial-creating-threads-and-multithr
  173. Thread Dump Analysis Tool / Method: https://stackoverflow.com/questions/3156434/thread-dump-analysis-tool-method
  174. Java - What is the Limit to the Number of Threads You Can Create?: https://dzone.com/articles/java-what-limit-number-threads
  175. Going inside Java’s Project Loom and virtual threads: https://blogs.oracle.com/javamagazine/going-inside-javas-project-loom-and-virtual-threads
  176. Multithreading in java with examples: https://beginnersbook.com/2013/03/multithreading-in-java/
  177. TutorialsPoint - Java Multi-Threading: https://www.tutorialspoint.com/java/java_multithreading.htm
  178. Java Multithreading example: https://appdividend.com/2019/05/29/java-multithreading-tutorial-with-example-multithreading-in-java/
  179. How to create a thread using lambda expressions in Java? : https://www.tutorialspoint.com/how-to-create-a-thread-using-lambda-expressions-in-java
  180. Java JDK 11 -- JavaDocs - Thread class: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/Thread.html (examples of the two primary methods 1. extends Thread, 2. implements Runnable
  181. Invoke method inside a Thread: https://stackoverflow.com/questions/39834848/invoke-method-inside-a-thread
  182. What is GC Log, thread dump and Heapdump?: https://blog.heaphero.io/2020/10/16/what-is-gc-log-thread-dump-and-heapdump/
  183. The jmap Utility: http://www.cs.gettysburg.edu/java/docs/technotes/guides/troubleshoot/tooldescr014.html
  184. Recommendations for a heap analysis tool for Java?: https://stackoverflow.com/questions/2064427/recommendations-for-a-heap-analysis-tool-for-java
  185. DZone - Java Heap Dump Analyzer overview: https://dzone.com/articles/java-heap-dump-analyzer-1
  186. Java Memory Analysis - Problems and Solutions (SLIDES): https://www.slideshare.net/MikhailMishaDmitriev/java-memory-analysis-problems-and-solutions?ref=https://jxray.com/
  187. Largest Contentful Paint: https://web.dev/largest-contentful-paint
  188. Beyond Speed - Why You Should Be Tracking Interactive Metrics: https://rigor.com/blog/tracking-interactive-metrics-speed
  189. The “Best” Performance Metrics? Start with These Six: https://rigor.com/blog/six-web-performance-metrics-to-watch
  190. Does the <script> tag position in HTML affects performance of the webpage?: https://stackoverflow.com/questions/4396849/does-the-script-tag-position-in-html-affects-performance-of-the-webpage
  191. JavaScript tags, performance and W3C: https://stackoverflow.com/questions/2812496/javascript-tags-performance-and-w3c
  192. Building a Culture of Performance in Springfield: The Simpsons as Metrics: https://rigor.com/blog/the-simpsons-as-performance-metric
  193. Viewability best practices: https://support.google.com/admanager/answer/6199883?hl=en
  194. Load google analytics javascript few seconds after the page has loaded: https://stackoverflow.com/questions/7930109/load-google-analytics-javascript-few-seconds-after-the-page-has-loaded
  195. Remove or Defer Blocking JS: https://developers.google.com/speed/docs/insights/BlockingJS
  196. Lazy load Google Analytics: https://gist.github.com/Nilpo/8129926
  197. Loading Third-Party JavaScript: https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/loading-third-party-javascript/
  198. How does Google Analytics affect mobile speed?: pagepipe.com/how-does-google-analytics-affect-mobile-page-speed/
  199. Google Analytics And The Page Load: https://www.simoahava.com/analytics/google-analytics-page-load/
  200. How to Append <script></script> in javascript?: https://stackoverflow.com/questions/9413737/how-to-append-script-script-in-javascript
  201. load scripts asynchronously: https://stackoverflow.com/questions/7718935/load-scripts-asynchronously#7719185
  202. Dynamically include Javascript within HTML body: https://techblog.synagila.com/2014/07/09/dynamically-include-javascript-within-html-body/
  203. Gotchas with dynamically adding script tags to HTML: https://www.danielcrabtree.com/blog/25/gotchas-with-dynamically-adding-script-tags-to-html
  204. Everything I Know About The Script Tag: https://eager.io/blog/everything-I-know-about-the-script-tag/
  205. The Curious Case of document.write: https://eager.io/blog/the-curious-case-of-document-write/
  206. JavaScript Madness -- Dynamic Script Loading: https://unixpapa.com/js/dyna.html
  207. The best way to load external JavaScript: https://humanwhocodes.com/blog/2009/07/28/the-best-way-to-load-external-javascript/
  208. How to insert <script> tag before </body>?: https://github.com/nuxt/nuxt.js/issues/2000
  209. Lazy-loading images: https://web.dev/lazy-loading-images/
  210. Browser-level image lazy-loading for the web: https://web.dev/browser-level-image-lazy-loading/
  211. Lazy Loading Images on the Web: https://developer.telerik.com/featured/lazy-loading-images-on-the-web/
  212. Lazy Loading Images: https://css-tricks.com/snippets/javascript/lazy-loading-images/
  213. Five Techniques to Lazy Load Images for Website Performance: https://www.sitepoint.com/five-techniques-lazy-load-images-website-performance/
  214. Lazy Loading with Intersection Observer Example: https://developers.google.com/web/fundamentals/performance/lazy-loading-guidance/images-and-video/ | DEMO
  215. Extend feature detection with isIntersecting: https://github.com/w3c/IntersectionObserver/issues/211
  216. Binding image lazy loading to new images inserted after ajax request: https://stackoverflow.com/questions/10035985/binding-image-lazy-loading-to-new-images-inserted-after-ajax-request
  217. Lazy Loading remastered -- Delay loading of images until user scrolls to them: https://appelsiini.net/projects/lazyload/
  218. YUI lib -- ImageLoader utility: https://yuilibrary.com/yui/docs/imageloader/
  219. Web.dev -- content-visibility - the new CSS property that boosts your rendering performance: https://web.dev/content-visibility/
  220. Content-visibility and Accessible Semantics: https://marcysutton.com/content-visibility-accessible-semantics
  221. More on content-visibility : https://css-tricks.com/more-on-content-visibility/
  222. Short note on content-visibility: hidden: https://html5accessibility.com/stuff/2020/08/25/short-note-on-content-visibility-hidden/
  223. Can multiple event listeners/handlers be added to the same element using javascript?: https://stackoverflow.com/questions/5411055/can-multiple-event-listeners-handlers-be-added-to-the-same-element-using-javascr
  224. Event listener performance with vanilla JS: https://gomakethings.com/event-listener-performance-with-vanilla-js/
  225. Cross-browser event listener with design patterns: https://web.archive.org/web/20120331200901/http://javascriptrules.com:80/2009/07/22/cross-browser-event-listener-with-design-patterns/
  226. Can we have multiple $ (document).ready() functions in jQuery? If yes, then what is the main advantage?: https://www.quora.com/Can-we-have-multiple-document-ready-functions-in-jQuery-If-yes-then-what-is-the-main-advantage
  227. jQuery - multiple $(document).ready …?: https://stackoverflow.com/questions/5263385/jquery-multiple-document-ready
  228. Can you have multiple $(document).ready(function(){ … }); sections?: https://stackoverflow.com/questions/1327756/can-you-have-multiple-document-readyfunction-sections
  229. Event Listeners Efficiency: https://stackoverflow.com/questions/41798755/event-listeners-efficiency
  230. Improving Javascript Performance with Event Delegation: http://blog.avenuecode.com/improving-javascript-performance-with-event-delegation
  231. Should I reduce event listeners by making functions more complex?: https://softwareengineering.stackexchange.com/questions/340355/should-i-reduce-event-listeners-by-making-functions-more-complex
  232. Optimizing javascript performance with many event listeners: https://stackoverflow.com/questions/29004245/optimizing-javascript-performance-with-many-event-listeners
  233. Does adding too many event listeners affect performance?: https://stackoverflow.com/questions/28627606/does-adding-too-many-event-listeners-affect-performance (benchmarks show "ID" attribute selectors faster than "class" attribute selectors)
  234. Event Delegation vs Direct Binding when adding complex elements to a page: https://stackoverflow.com/questions/8827430/event-delegation-vs-direct-binding-when-adding-complex-elements-to-a-page
  235. Speed Up Scroll Events with Passive Event Listeners: https://alligator.io/js/speed-up-scroll-events/
  236. Optimizing javascript performance with many event listeners: https://stackoverflow.com/questions/29004245/optimizing-javascript-performance-with-many-event-listeners
  237. Verizon buys Yahoo! for $4.83 billion USD: https://techcrunch.com/2016/07/25/verizon-buys-yahoo-for-4-83-billion/
  238. yslow Yahoo! dev network (old link, now redirects): http://developer.yahoo.com/yslow/
  239. Find And Fix The Hogs On Your Site Using YSlow Firefox Add-On: https://www.maketecheasier.com/find-and-fix-the-hogs-on-your-site-using-yslow-firefox-add-on/
  240. YSlow is broken after update to Firefox 36: https://github.com/marcelduran/yslow/issues/142
  241. Yslow won't automatically open: https://groups.google.com/forum/#!topic/showslow/kQ4qSnPeSSc
  242. Firebug is not working with Firefox version 50.0:http://stackoverflow.com/questions/40800173/firebug-is-not-working-with-firefox-version-50-0/42218128#42218128 (on FF, yslow depends on Firebug)
  243. Unifying Firebug & Firefox DevTools -- Unifying Firebug & Firefox DevTools: https://blog.getfirebug.com/2016/06/07/unifying-firebug-firefox-devtools/ (as of FF 50.x+ Firebug & FF Dev Tools are one and the same) | SRC
  244. Install an older version of Firefox: https://support.mozilla.org/t5/Install-and-Update/Install-an-older-version-of-Firefox/ta-p/1564 (pre-v36.x probably best bet for yslow support)
  245. Google's PageSpeed alternative: https://developers.google.com/speed/pagespeed/insights/
  246. How to enable YSlow in Firefox?: http://stackoverflow.com/questions/7508548/how-to-enable-yslow-in-firefox
  247. A YSlow Alternative for Making Web Pages Faster: http://www.maxcdn.com/blog/coach-yslow-alternative/
  248. Google Experimental Speed Report Goes Public: https://www.mediapost.com/publications/article/342833/google-experimental-speed-report-goes-public.html
  249. Page Load Times, Technology Are Major Ranking Factors In Google Search Results: https://www.mediapost.com/publications/article/335987/page-load-times-technology-are-major-ranking-fact.html
  250. Continuous Performance Analysis with Lighthouse and Jenkins: https://kitconcept.com/blog/continuous-performance-analysis-with-lighthouse-and-jenkins/
  251. Lighthouse Bot - TravisCI & GitHub recipe: https://github.com/GoogleChromeLabs/lighthousebot
  252. Integrate Google Lighthouse into GitLab's CI to evaluate frontend code performance: https://gitlab.com/gitlab-org/gitlab-ce/issues/33392
  253. What do Lighthouse Scores look like across the web?: https://www.tunetheweb.com/blog/what-do-lighthouse-scores-look-like-across-the-web/
  254. How to do web (front-end) performance testing with Google Lighthouse & Cypress: https://techbeacon.com/app-dev-testing/how-do-web-performance-testing-google-lighthouse-cypress
  255. 9 tools to help you with Java Performance Tuning: https://blog.idrsolutions.com/2014/06/java-performance-tuning-tools/
  256. Hypertext Transfer Protocol -- HTTP/1.1 (SECTION on "Caching"): http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
  257. Hypertext Transfer Protocol -- HTTP/1.1 (SECTION on "Header Field Definitions"): http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
  258. Web Caching: http://www.cisco.com/web/about/ac123/ac147/ac174/ac199/about_cisco_ipj_archive_article09186a00800c8903.html
  259. POWER AND MEMORY OPTIMIZATION TECHNIQUES IN EMBEDDED SYSTEMS DESIGN: http://etd.lsu.edu/docs/available/etd-07122005-120721/unrestricted/Allam_dis.pdf
  260. MIT - Spring2010 course on Memory Optimization: https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-035-computer-language-engineering-spring-2010/lecture-notes/MIT6_035S10_lec18.pdf
  261. Front-End Performance Checklist 2019: https://www.smashingmagazine.com/2019/01/front-end-performance-checklist-2019-pdf-pages/#download-the-checklist
  262. The “Best” Performance Metrics? Start With These Six: https://rigor.com/blog/six-web-performance-metrics-to-watch
  263. Swap usage: http://www.tomshardware.com/answers/id-2141305/swap-usage.html
  264. High Swap Utilization, lot's of RAM free: https://developer.jboss.org/thread/70542?_sscc=t
  265. How To -- Check SWAP utilization in Linux: https://www.cyberciti.biz/faq/linux-check-swap-usage-command/
  266. Swap utilization going over: https://www.linux.com/answers/swap-utilization-going-over
  267. 8 Useful Commands to Monitor Swap Space Usage in Linux: https://www.tecmint.com/commands-to-monitor-swap-space-usage-in-linux/
  268. Introduction to Profiling Java Applications in NetBeans IDE: https://netbeans.org/kb/docs/java/profiler-intro.html
  269. Eclipse Java Profiler: http://stackoverflow.com/questions/2713940/eclipse-java-profiler
  270. Java Application Profiling using TPTP: http://www.eclipse.org/articles/Article-TPTP-Profiling-Tool/tptpProfilingArticle.html
  271. Deep monitoring with JMX: https://blogs.oracle.com/java-platform-group/entry/deep_monitoring_with_jmx
  272. J2SE - About Java Mission Control: https://docs.oracle.com/javacomponents/jmc-5-4/jmc-user-guide/intro.htm#CEGIAGIG
  273. Troubleshoot Java Mission Control: https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr002.html
  274. Simplify Tomcat/JVM Monitoring with Mission Control (JMX) or VisualVM: https://rahulvishwakarma.wordpress.com/2016/06/23/simplify-tomcatjvm-monitoring-with-mission-control-jmx-or-visualvm/
  275. Code-based Memory Optimizations: https://www.cs.utexas.edu/~pingali/CS395T/2013fa/lectures/MemoryOptimizations_2013.pdf
  276. Script Tag - async & defer: https://stackoverflow.com/questions/10808109/script-tag-async-defer
  277. Lead Time vs Cycle Time: https://simplicable.com/new/lead-time-vs-cycle-time
  278. Lead Time vs Cycle Time: https://leanandkanban.wordpress.com/2009/04/18/lead-time-vs-cycle-time/
  279. Kanban -- Definition of Lead Time and Cycle Time: https://stefanroock.wordpress.com/2010/03/02/kanban-definition-of-lead-time-and-cycle-time/
  280. HBS Toolkit - Basic Operations Self-Instructional Workbook: https://web.archive.org/web/20130308112606/https://hbswk.hbs.edu/archive/1460.html
  281. How ConnectALL Calculates Lead Time and why measuring Cycle Time is important: https://www.brighttalk.com/webcast/18060/436929

See Also

Agile transformation | DevOps | Dashboard | Performance Testing | HTML | Image | Audio | Video | Web Service | DBMS | Monitoring | IPS | IDS | Security | Productivity | AMP