在线时间:8:00-16:00
迪恩网络APP
随时随地掌握行业动态
扫描二维码
关注迪恩网络微信公众号
开源软件名称:allinurl/goaccess开源软件地址:https://github.com/allinurl/goaccess开源编程语言:C 86.5%开源软件介绍:GoAccessWhat is it?GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal on *nix systems or through your browser. It provides fast and valuable HTTP statistics for system administrators that require a visual server report on the fly. More info at: https://goaccess.io. FeaturesGoAccess parses the specified web log file and outputs the data to the X terminal. Features include:
Nearly all web log formats...GoAccess allows any custom log format string. Predefined options include, but not limited to:
Why GoAccess?GoAccess was designed to be a fast, terminal-based log analyzer. Its core idea is to quickly analyze and view web server statistics in real time without needing to use your browser (great if you want to do a quick analysis of your access log via SSH, or if you simply love working in the terminal). While the terminal output is the default output, it has the capability to
generate a complete, self-contained, real-time You can see it more of a monitor command tool than anything else. InstallationBuild from releaseGoAccess can be compiled and used on *nix systems. Download, extract and compile GoAccess with:
Build from GitHub (Development)
Build in isolated containerYou can also build the binary for Debian based systems in an isolated container environment to prevent cluttering your local system with the development libraries:
DistributionsIt is easiest to install GoAccess on GNU+Linux using the preferred package manager of your GNU+Linux distribution. Please note that not all distributions will have the latest version of GoAccess available. Debian/Ubuntu
Note: It is likely this will install an outdated version of GoAccess. To make sure that you're running the latest stable version of GoAccess see alternative option below. Official GoAccess Debian & Ubuntu repository
Note:
Fedora
Arch
Gentoo
OS X / Homebrew
FreeBSD
OpenBSD
openSUSE
OpenIndiana
pkgsrc (NetBSD, Solaris, SmartOS, ...)
WindowsCowAxess is a GoAccess implementation for Windows systems. It is a packaging of GoAccess, Cygwin and many other related tools to make it a complete and ready-to-use solution for real-time web log analysis, all in a 4 MB package. If you prefer to go the more tedious route, GoAccess can be used in Windows through Cygwin. See Cygwin's packages. Or through the GNU+Linux Subsystem on Windows 10. Distribution PackagesGoAccess has minimal requirements, it's written in C and requires only ncurses. However, below is a table of some optional dependencies in some distros to build GoAccess from source.
Note: You may need to install build tools like DockerA Docker image has been updated, capable of directing output from an access log. If you only want to output a report, you can pipe a log from the external environment to a Docker-based process:
OR real-time
You can read more about using the docker image in DOCKER.md. StorageDefault Hash TablesIn-memory storage provides better performance at the cost of limiting the dataset size to the amount of available physical memory. GoAccess uses in-memory hash tables. It has very good memory usage and pretty good performance. This storage has support for on-disk persistence as well. Command Line / Config OptionsSee options that can be supplied to the command or
specified in the configuration file. If specified in the configuration file, long
options need to be used without prepending Usage / ExamplesNote: Piping data into GoAccess won't prompt a log/date/time configuration dialog, you will need to previously define it in your configuration file or in the command line. Getting StartedTo output to a terminal and generate an interactive report:
To generate an HTML report:
To generate a JSON report:
To generate a CSV file:
GoAccess also allows great flexibility for real-time filtering and parsing. For instance, to quickly diagnose issues by monitoring logs since goaccess was started:
And even better, to filter while maintaining opened a pipe to preserve
real-time analysis, we can make use of
or to parse from the beginning of the file while maintaining the pipe opened and applying a filter
Multiple Log filesThere are several ways to parse multiple logs with GoAccess. The simplest is to pass multiple log files to the command line:
It's even possible to parse files from a pipe while reading regular files:
Note: the single dash is appended to the command line to let GoAccess know that it should read from the pipe. Now if we want to add more flexibility to GoAccess, we can use
Note: On Mac OS X, use Real-time HTML outputsGoAccess has the ability the output real-time data in the HTML report. You can even email the HTML file since it is composed of a single file with no external file dependencies, how neat is that! The process of generating a real-time HTML report is very similar to the
process of creating a static report. Only
To view the report you can navigate to By default, GoAccess will use the host name of the generated report. Optionally, you can specify the URL to which the client's browser will connect to. See FAQ for a more detailed example.
By default, GoAccess listens on port 7890, to use a different port other than 7890, you can specify it as (make sure the port is opened):
And to bind the WebSocket server to a different address other than 0.0.0.0, you can specify it as:
Note: To output real time data over a TLS/SSL connection, you need to use
FilteringWorking with datesAnother useful pipe would be filtering dates out of the web log The following will get all HTTP requests starting on
or using relative dates such as yesterdays or tomorrows day:
If we want to parse only a certain time-frame from DATE a to DATE b, we can do:
If we want to preserve only certain amount of data and recycle storage, we can keep only a certain number of days. For instance to keep & show the last 5 days:
Virtual hostsAssuming your log contains the virtual host field. For instance:
And you would like to append the virtual host to the request in order to see which virtual host the top urls belong to:
To do the same, but also use real-time filtering and parsing:
To exclude a list of virtual hosts you can do the following:
Files, status codes and botsTo parse specific pages, e.g., page views,
Note,
Or to parse a specific status code, e.g., 500 (Internal Server Error):
Or multiple status codes, e.g., all 3xx and 5xx:
And to get an estimated overview of how many bots (crawlers) are hitting your server:
TipsAlso, it is worth pointing out that if we want to run GoAccess at lower priority, we can run it as:
and if you don't want to install it on your server, you can still run it from your local machine!
Note: SSH requires TroubleshootingWe receive many questions and issues that have been answered previously.
Incremental log processingGoAccess has the ability to process logs incrementally through its internal storage and dump its data to disk. It works in the following way:
NOTESGoAccess keeps track of inodes of all the files processed (assuming files will stay on the same partition), in addition, it extracts a snippet of data from the log along with the last line parsed of each file and the timestamp of the last line parsed. e.g., inode:29627417|line:20012|ts:20171231235059 First, it compares if the snippet matches the log being parsed, if it does, it assumes the log hasn't changed drastically, e.g., hasn't been truncated. If the inode does not match the current file, it parses all lines. If the current file matches the inode, it then reads the remaining lines and updates the count of lines parsed and the timestamp. As an extra precaution, it won't parse log lines with a timestamp ≤ than the one stored. Piped data works based off the timestamp of the last line read. For instance, it will parse and discard all incoming entries until it finds a timestamp >= than the one stored. Examples
then, load it with
To read persisted data only (without parsing new data)
ContributingAny help on GoAccess is welcome. The most helpful way is to try it out and give feedback. Feel free to use the Github issue tracker and pull requests to discuss and submit code changes. Enjoy! |
2023-10-27
2022-08-15
2022-08-17
2022-09-23
2022-08-13
请发表评论