17th-Ranger-Battalion-ORG/RangerMetrics
7
1
Fork 0
Code Issues 4 Pull Requests Packages Projects Releases Wiki Activity

MAJOR rework

Browse Source 
- improves a lot on the Go side including better config and logging libraries (including log rotation), better internal package distribution, and new a3go functionality to make data transfer more performant
- SQF side preprocessing of capture data is now minimal - arrays in hashmap format are sent directly to the extension and parsed there to minimize game impact
- CBA custom events are implemented in a better fashion
- README update
- license change
- with performance improvements, the deep customization of integrated metric gathering is removed in return to a single refreshRateMs, defining the interval at which core metrics are captured
- peeled back the list of core metrics to the core information used in troubleshooting and benchmarking
This commit is contained in:
IndigoFox
2023-10-10 00:44:50 -07:00
parent cf45d6b263
commit dc822c4c93
75 changed files with 4335 additions and 3347 deletions
Download Patch File Download Diff File Expand all files Collapse all files

160
README.md
View File

@@ -1,4 +1,4 @@
# RangerMetrics - Arma3 InfluxDB Metrics
# IFX Metrics - Arma3 Server Performance
This addon is designed to capture data from Arma3 and send it to InfluxDB. The extension is written in Golang and uses a non-blocking Write API to send data to InfluxDB.
@@ -8,50 +8,152 @@ The data in Influx can be used to create dashboards in Grafana.
## Setup
### Settings.json
### ifxmetrics.config.json
Configure the options in settings.json.
Copy the ifxmetrics.config.example.json as ifxmetrics.config.json and edit the values to match your environment.
### Custom CBA Events to Log Metrics
The `cbaEventHandlers` sectio0n of the JSON configuration is used to enable custom server-side event listeners. In this way, you can capture measurements from custom scripts and send it to InfluxDB.
> You can read more about the data format that InfluxDB uses [here](https://docs.influxdata.com/influxdb/v2/get-started/write/#line-protocol-elements).
In this implementation, the bucket and measurement are defined in the config file. A bucket is created if it doesn't exist when you first send data to it, so you don't need to create buckets manually. As a tradeoff for security's sake, **the ability to define new buckets to send data to is not exposed to the scripter**.
As a brief note, tags are indexed while fields are not. This means that categorization generally happens at the tag level, and every field you submit will be categorized by each tag you provide with that send.
In the below example, the `milsimServerEfficiency` event handler will send data to `server_performance` (bucket) -> `milsim_server_efficiency` (measurement). The fields durationMs and vehicleCount will be sent as fields -- each can be easily filtered by the tags provided.
```json
// configuration file
// ...
"cbaEventHandlers": {
"milsimServerEfficiency": {
"name": "milsimServerEfficiency",
"description": "EVENTHANDLER. Tracks the efficiency of the server.",
"enabled": true,
"bucket": "server_performance",
"measurement": "milsim_server_efficiency"
}
}
// ...
```
```sqf
// your custom SQF script
fnc_getServerEfficiency = {
private _startTime = diag_tickTime;
// make and delete 99 vehicles
for "_i" from 0 to 99 do {
_vehicle = createVehicle ["B_Heli_Transport_01_F", findEmptyPosition [[0,0,0], 100, 10], [], 0, "FLY"];
deleteVehicle _vehicle;
};
// get the time it took to make and delete 99 vehicles
private _endTime = diag_tickTime;
private _timeTaken = _endTime - _startTime;
// send the data to InfluxDB
["milsimServerEfficiency", [
[ // tags in hash format. must be string values!
["missionPhase", "init"],
["missionName", "My Mission"],
["missionType", "COOP"],
["serverName", "My Server"]
],
[ // fields in hash format. can be any type
["durationMs", _timeTaken],
["vehicleCount", 99]
]
]] call CBA_fnc_serverEvent;
};
```
---
### InfluxDB
InfluxDB is a time series database. It is used to store data points with a timestamp.
#### Required Buckets
To set up a basic Docker instance with data persisted in your terminal's current working directory, run the following command:
- mission_data
- player_performance
- server_performance
- server_events
- soldier_ammo
```bash
docker run -d -p 8086:8086 -v $PWD/influxdb/data:/var/lib/influxdb2 -v $PWD/influxdb/config:/etc/influxdb2 -e DOCKER_INFLUXDB_INIT_MODE=setup -e DOCKER_INFLUXDB_INIT_USERNAME=myuser -e DOCKER_INFLUXDB_INIT_PASSWORD=dfaow3ho9i7funa0w3nv -e DOCKER_INFLUXDB_INIT_ORG=ifx-metrics -e DOCKER_INFLUXDB_INIT_BUCKET=test-bucket -e DOCKER_INFLUXDB_INIT_RETENTION=1w -e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=f0982q3ahfu8yawbo27w8fb986ba90b0wb2f influxdb:latest
```
This will create a new InfluxDB instance with the following credentials:
| Username | Password |
| --- | --- |
| myuser | dfaow3ho9i7funa0w3nv |
The database will be created with the following settings:
| Organization | Bucket | Retention |
| --- | --- | --- |
| ifx-metrics | test-bucket | 1 week |
The admin token, which is used with the InfluxDB API and CLI for administration, is:
| Token |
| --- |
| f0982q3ahfu8yawbo27w8fb986ba90b0wb2f |
Your InfluxDB instance will be available at <http://localhost:8086>.
### Grafana
Grafana is a dashboarding tool. It is used to display the data from InfluxDB. Import the dashboard from the json file in the root of this addon and set up your datasources appropriately.
Grafana is a dashboarding tool. It is used to display the data from InfluxDB. A sample dashboard is provided as a starting point.
---
## Usage
## BUILDING
### Ingame
Set an environment variable in your terminal with the desired extension build version. It defaults to "DEVELOPMENT".
#### Toggle Capture On/Off
Running the following commands in Server Exec will toggle the capture on or off for the server and any Headless Clients. Capture loops will still occur but will exit almost immediately.
Change the last parameter to false to ONLY target the server when run as Server Exec.
*This may not apply to raw Event Handler data which goes under `server_events`*
```sqf
// ON
missionNamespace setVariable ["RangerMetrics_run", true, true];
// OFF
missionNamespace setVariable ["RangerMetrics_run", false, true];
```powershell
# powershell
$IFXMETRICS_BUILD_VER = "0.1.0-$(Get-Date -Format 'yyyyMMdd')-$(git rev-parse --short HEAD)"
```
#### Reload Settings.json and recreate all capture loops
```bash
# bash
export IFXMETRICS_BUILD_VER="0.1.0-$(date -u '+%Y%m%d')-$(git rev-parse --short HEAD)"
```
To reload everything while in a game, run `"RangerMetrics" callExtension "initExtension";` in Global Exec. This will disconnect any database connections and reset state. Running it Global Exec will cause any client with the addon to run it, which includes Headless Clients.
### EXTENSION: COMPILING FOR WINDOWS
When the extension is finished, it will notify Arma via a callback. The addon will then __automatically__ reinitialize the extension, to include fetching the new settings, tearing down existing captures, and re-setting up captures with the new settings.
Run this from the project root.
```powershell
docker pull x1unix/go-mingw:1.20
# Compile x64 Windows DLL
docker run --rm -it -v ${PWD}:/go/work -w /go/work -e GOARCH=amd64 -e CGO_ENABLED=1 x1unix/go-mingw:1.20 go build -o ./ifxmetrics_x64.dll -buildmode=c-shared -ldflags "-w -s main.EXTENSION_VERSION=${IFXMETRICS_BUILD_VER}" ./extension/cmd
# Compile x86 Windows DLL
docker run --rm -it -v ${PWD}:/go/work -w /go/work -e GOARCH=386 -e CGO_ENABLED=1 x1unix/go-mingw:1.20 go build -o ./ifxmetrics.dll -buildmode=c-shared -ldflags "-w -s main.EXTENSION_VERSION=${IFXMETRICS_BUILD_VER}" ./extension/cmd
# Compile x64 Windows EXE
docker run --rm -it -v ${PWD}:/go/work -w /go/work -e GOARCH=amd64 -e CGO_ENABLED=1 x1unix/go-mingw:1.20 go build -o ./ifxmetrics_x64.exe -ldflags "-w -s main.EXTENSION_VERSION=${IFXMETRICS_BUILD_VER}" ./extension/cmd
```
### EXTENSION: COMPILING FOR LINUX
Run this from the project root.
```powershell
docker build -t indifox926/build-a3go:linux-so -f ./build/Dockerfile.build .
# Compile x64 Linux .so
docker run --rm -it -v ${PWD}:/app -e GOOS=linux -e GOARCH=amd64 -e CGO_ENABLED=1 -e CC=gcc indifox926/build-a3go:linux-so go build -o ./ifxmetrics_x64.so -linkshared -ldflags "-w -s main.EXTENSION_VERSION=${IFXMETRICS_BUILD_VER}" ./extension/cmd
# Compile x86 Linux .so
docker run --rm -it -v ${PWD}:/app -e GOOS=linux -e GOARCH=386 -e CGO_ENABLED=1 -e CC=gcc indifox926/build-a3go:linux-so go build -o ./ifxmetrics.so -linkshared -ldflags "-w -s main.EXTENSION_VERSION=${IFXMETRICS_BUILD_VER}" ./extension/cmd
```
### ADDON: COMPILE USING HEMTT
Download the [HEMTT binary](https://github.com/BrettMayson/HEMTT/releases/latest) and place it in the project root. The configuration inside will be read by the HEMTT exe and defines the build process.
```powershell
./hemtt.exe release
```