Logging
From BC$ MobileTV Wiki
Logging is the process of recording the usage of a specific hardware or software component. A number of tools exist for logging, some focused on server (access-based) logging, and some based on individual program (class/flow-based) logging.
Contents
Specifications
- W3C -- Extended Log File Format: https://www.w3.org/TR/WD-logfile
Tools
APM tools also come in quite handy for quickly finding the "needle in the haystack" of root cases of errors/problems from within your logs.
Java
- System.logger (defaults to "JUL" as implementation) [1]
- java.util.logging (JUL)[2]
- slf4j: http://web.archive.org/web/20150512172426/http://www.slf4j.org/ | GUIDE (wrapper/spec for interfacing to any chosen logging framework: java.util.logging, commons-logging, log4j, logback, etc)
- Logback: http://logback.qos.ch/ | GUIDE
- log4j: http://logging.apache.org/log4j/ | GUIDE (application error/exception/warning logging)
- commons-logging: http://commons.apache.org/ | GUIDE
- Google - Flogger: https://github.com/google/flogger[3]
- Apple - Garbage Collection Graph Collector (by Apple): https://github.com/apple/GCGC [4]
PHP
- log4php: http://logging.apache.org/log4php/
- AWStats -- Free real-time logfile analyzer to get advanced statistics (server log): http://awstats.sourceforge.net/
- PHP Log Analyzer (server log): http://sourceforge.net/projects/phploganalyzer/
- Scribe PHP logging: http://www.ruturaj.net/scribe-php-logging
- PHP logging class v. 1.2: http://urbanoalvarez.es/blog/2008/03/21/php-logging-class/
Log Analyzers
Splunk
- Splunk: https://www.splunk.com
- NodeJS Backend Collector for Client-Side Data (GoogleAnalytics "local" alternative): https://www.splunk.com/en_us/blog/tips-and-tricks/still-using-3rd-party-web-analytics-providers-build-your-own-using-splunk.html | SRC (still using 3rd party web analytics providers? Build your own using Splunk!)
Kafka
- Apache Kafka: https://kafka.apache.org/
Resources
- The Complete Guide to the ELK Stack: http://logz.io/learn/complete-guide-elk-stack/
- log4j-viewer (Eclipse IDE): http://code.google.com/p/log4j-viewer/
- Log4j setups: http://www.slf4j.org/images/legacy.png
- Sling Commons Log components: http://sling.apache.org/documentation/development/logging.html
- AEM Chrome Plug-in - Log Tracer: https://adobe-consulting-services.github.io/acs-aem-tools/aem-chrome-plugin/log-tracer/
Tutorials
- Java Utils Logging (JUL) framework tutorial: http://www3.ntu.edu.sg/home/ehchua/programming/java/JavaLogging.html
- Java Logging - where is my log file?: https://stackoverflow.com/questions/9457976/java-logging-where-is-my-log-file
- Logging Messages in JAVA with commons-logging (or java.util.log): http://www.javapractices.com/topic/TopicAction.do?Id=143 (solid reference for ACL lib)
- Log4J Quick Guide: http://www.tutorialspoint.com/log4j/log4j_quick_guide.htm
- Log4J Levels: http://www.allapplabs.com/log4j/log4j_levels.htm[13][14]
- Log4J Tutorial: http://www.vaannila.com/log4j/log4j-tutorial/log4j-tutorial.html
- Logging with log4j: http://www.laliluna.de/download/log4j-tutorial-en.pdf
- Sawing through the Java Loggers: http://architects.dzone.com/articles/sawing-through-java-loggers
- How to Set up Java Logging: http://www.forward.com.au/javaProgramming/javaGuiTips/javaLogging.html
- Setting the Logging Configuration Programmatically: http://docs.oracle.com/cd/E19340-01/820-6767/gcbkk/index.html (change log filename/location and level, etc in Java code)
- The Truth of Java Exceptions - What's Really Going on Under the Hood?: https://dzone.com/articles/the-truth-of-java-exceptions-whats-really-going-on
- How to use Apache's Log4J RollingFileAppender: http://mark.koli.ch/2009/08/howto-using-apaches-log4j-rollingfileappender-an-example.html
- How do I create a rolling log files (using only java.util.logging default logger)?: http://www.kodejava.org/examples/460.html
- java.util.logging Example: https://examples.javacodegeeks.com/core-java/util/logging/java-util-logging-example/
- Configure java.util.logging programmatically: http://blog.cag.se/?p=835
- Use a Reverse Grep to Filter "tail -f" Output from command-line: http://www.howtogeek.com/50829/use-a-reverse-grep-to-filter-tail-f-output/ (omit certain text values from log and find next/other errors)
- 7 Linux Tail Command Examples and How it Helps Monitor Logs: https://linoxide.com/linux-command/linux-tail-command/
- TAIL GREP APACHE LOG AND DNS REVERSE LOOK UP IN REAL TIME: http://jotham.wordpress.com/2013/11/13/tail-grep-apache-log-and-reverse-look-up-ip-in-real-time/ (rolling list of reverse lookups matching the ip that requested)
- Tail a Log File on Windows & Linux with several different tools/options: https://stackify.com/13-ways-to-tail-a-log-file-on-windows-unix/ (vi, etc)
- How do I copy a file/folder from another user's home directory in Linux?: https://superuser.com/questions/353565/how-do-i-copy-a-file-folder-from-another-users-home-directory-in-linux
- Saving Your Putty Sessions To A Log File: https://www.packet6.com/saving-your-putty-sessions-to-a-log-file/
- How you can improve your workflow using the JavaScript console: https://medium.freecodecamp.org/how-you-can-improve-your-workflow-using-the-javascript-console-bdd7823a9472
- The Log - What every software engineer should know about real-time data's unifying abstraction: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying
- 9 Best Practices to Handle Exceptions in Java: https://stackify.com/best-practices-exceptions-java/[15]
- Logging Levels 101: https://stackify.com/logging-levels-101/
- SLF4J -- 10 Reasons Why You Should Be Using It: https://stackify.com/slf4j-java/
- Java Logging Frameworks -- log4j .vs. logback .vs. log4j2: https://stackify.com/compare-java-logging-frameworks/
- Saving Time with Structured Logging: https://reflectoring.io/structured-logging/
- FF Dev Tools -- Set a logpoint: https://developer.mozilla.org/en-US/docs/Tools/Debugger/Set_a_logpoint
- Stop Using console.log() In JavaScript: https://medium.com/javascript-in-plain-english/stop-using-console-log-in-javascript-d29d6c24dc26
- Logging Guide for Hibernate 4, 5 & 6 – Use the right config for Dev & Prod: https://thorben-janssen.com/hibernate-logging-guide/
- Correlation ID for Logging in Microservices: https://dzone.com/articles/correlation-id-for-logging-in-microservices
- Improved Java Logging with Mapped Diagnostic Context (MDC): https://www.baeldung.com/mdc-in-log4j-2-logback
- MDC - A Better Way of Logging: https://dzone.com/articles/mdc-better-way-of-logging-1
- Understanding Java Garbage Collection Logging - What Are GC Logs and How To Analyze Them: https://sematext.com/blog/java-garbage-collection-logs/
- Garbage Collection Logging to a File in Java: https://www.baeldung.com/java-gc-logging-to-file
External Links
- wikipedia: Log file
- wikipedia: Common Log Format
- wikipedia: Extended Log Format
- wikipedia: Data logger
- wikipedia: Security log
- wikipedia: Syslog
- wikipedia: Audit trail
- wikipedia: Log management
- wikipedia: Log analysis
- wikipedia: Web log analysis software
- Short introduction to log4j: http://logging.apache.org/log4j/1.2/manual.html
- A simple example of log4j: http://www.roseindia.net/tutorials/log4j/log4j.shtml
- very large comprehensive log4j.properties exmaple: http://www.johnmunsch.com/projects/Presentations/docs/Log4J/log.properties
- Sample log4j.properties file: http://www.benmccann.com/dev-blog/sample-log4j-properties-file/
- Top 10 tips on logging in Java: http://javarevisited.blogspot.com/2011/05/top-10-tips-on-logging-in-java.html
- There are things to love about java.util.logging: http://blogs.bytecode.com.au/glen/2006/04/07/there-are-things-to-love-about-java-util-logging.html
- Java UTIL logging configs: http://vigilog.sourceforge.net/logconfig.html
- Top 5 Java Logging Frameworks and Tools (2017): https://dzone.com/articles/java-logging-frameworks-and-tools
- 9 Best Practices to Handle Exceptions in Java: https://stackify.com/best-practices-exceptions-java/
- Please stop using console.log(), it’s broken: https://hackernoon.com/please-stop-using-console-log-its-broken-b5d7d396cf15
- 3 Use Cases Where Structured Log Data Really Helps: https://reflectoring.io/structured-log-data/
- j-text-utils: https://code.google.com/archive/p/j-text-utils/
- Java Command Line - How to create table in a Command line application in Java?: https://www.logicbig.com/how-to/code-snippets/jcode-java-cmd-command-line-table.html
- How to Print data to JTables in Swing: https://docs.oracle.com/javase/tutorial/uiswing/misc/printtable.html
- How to Pretty Print JSON on the Command Line: http://bigdatums.net/2016/10/26/how-to-pretty-print-json-on-the-command-line/
- How To Pretty Print and Format XML in Command Line Linux?: https://www.poftut.com/pretty-print-format-xml-command-line-linux/
- Top Five Reasons to use Crashlytics (for "Mobile App remote log" analysis): https://firebase.googleblog.com/2020/07/top-five-reasons-to-use-crashlytics.html
- The Case Against Logging: https://blog.sebastian-daschner.com/entries/the-case-against-logging
References
- ↑ Java's new "System Logger" API: https://blog.frankel.ch/system-logger/ | SRC
- ↑ What are the latest options in Java logging frameworks?: https://stackoverflow.com/questions/2533614/what-are-the-latest-options-in-java-logging-frameworks
- ↑ Google Releases New Java Logging Framework: https://www.infoq.com/news/2019/04/java-logging-framework-flogger
- ↑ Apple Open Sources GCGC: a Tool to Analyze Java GC Logs: https://www.infoq.com/news/2022/01/apple-gcgc-java/
- ↑ How to exclude a list of values for a field?: https://answers.splunk.com/answers/48398/how-to-exclude-a-list-of-values-for-a-field.html
- ↑ About fields -- How to extract/define and save fields for future lookups: http://docs.splunk.com/Documentation/Splunk/7.1.1/Knowledge/Aboutfields
- ↑ Splunk acquires VictorOps to take it – and you – into site reliability engineering: https://www.theregister.co.uk/2018/06/19/splunk_victorops_acquisition_digested/
- ↑ Why Build Your Own (BYO) Client-Side Analytics?: https://www.splunk.com/en_us/blog/tips-and-tricks/still-using-3rd-party-web-analytics-providers-build-your-own-using-splunk.html
- ↑ Splunkbase -- add-on marketplace - Splunk App for Web Analytics: https://splunkbase.splunk.com/app/2699/
- ↑ Splunk your GoogleAnalytics: https://www.splunk.com/en_us/blog/conf-splunklive/splunk-your-google-analytics.html
- ↑ Splunk for GoogleAnalytics: https://conf.splunk.com/files/2016/slides/splunk-for-google-analytics-how-to-gain-more-insight-into-site-performance-and-user-activity-by-correlating-google-analytics-data-within-splunk-software.pdf (the "if you can't beat'em join'em" approach)
- ↑ Splunk docs -- trendline: https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Trendline
- ↑ Log4J Levels - explanation of differences between the five logging levels: http://javapapers.com/log4j/log4j-levels/
- ↑ Using Logging Levels Consistently: https://reflectoring.io/logging-levels/
- ↑ Java Tip 134 -- When catching exceptions, don't cast your net too wide: https://www.infoworld.com/article/2077500/java-tip-134--when-catching-exceptions--don-t-cast-your-net-too-wide.html
- ↑ Java Garbage Collection Logging: https://support.sonatype.com/hc/en-us/articles/360000870607-Java-Garbage-Collection-Logging
- ↑ Understanding the Java Garbage Collection Log: https://dzone.com/articles/understanding-garbage-collection-log
- ↑ How to Enable Garbage Collection (GC) Logging: https://confluence.atlassian.com/confkb/how-to-enable-garbage-collection-gc-logging-300813751.html
- ↑ Output in a table format in Java's System.out: https://stackoverflow.com/questions/2745206/output-in-a-table-format-in-javas-system-out
- ↑ Formatting Java Output Like a Table: https://stackoverflow.com/questions/15052235/formatting-java-output-like-a-table
- ↑ Nicely Formatted Tabular Output in Java: https://web.archive.org/web/20150221143732/https://www.ksmpartners.com/2013/08 /nicely-formatted-tabular-output-in-java/
