📊 Project: Extracting and Aggregating Metrics from Server Logs

This project will guide you through building a powerful and efficient server log parser. It's a classic real-world problem that perfectly showcases the advantages of uCalc's token-aware Transformer over traditional tools like Regular Expressions or manual string splitting.

The Goal: From Raw Logs to Actionable Insights

Our objective is to take a raw, unstructured log file and transform it into a structured summary of key performance indicators (KPIs). We want to answer questions like:

• How many total requests were processed?
• What was the error rate?
• What was the average response time?
• What was the peak response time?

The Input Data

Here is a sample of the access.log content we will be working with. It contains a mix of successful requests and errors, with inconsistent spacing.

2024-10-26 10:00:05 INFO    192.168.1.10   "GET /api/users HTTP/1.1" 200 15ms2024-10-26 10:00:06 INFO    192.168.1.15   "GET /api/products HTTP/1.1" 200 22msThis is a malformed line and should be ignored.2024-10-26 10:00:07 ERROR   192.168.1.22   "POST /api/login HTTP/1.1" 500 120ms2024-10-26 10:00:08 INFO    192.168.1.10   "GET /api/users/1 HTTP/1.1" 200 8ms

The Strategy: A Single, Powerful Rule

We will use a single Transformer rule to process the entire log file. This rule will be designed to:

1. Match a Valid Log Line: The pattern will capture all the relevant fields: timestamp, log level, IP address, request details, status code, and response time.
2. Aggregate Metrics with {@Exec}: The replacement part of the rule will not produce any text. Instead, it will use a series of {@Exec} directives to update a set of counter variables for each line it processes. This is a highly efficient way to perform calculations as a side effect of a transformation.
3. Ignore Invalid Lines: The pattern will be specific enough that it won't match malformed lines, which will be gracefully ignored.

Step 1: Setting Up the Metric Variables

First, we need to define the variables in our uCalc instance that will store the aggregated metrics. We initialize them all to zero.

csharp
uc.DefineVariable("request_count = 0");
uc.DefineVariable("error_count = 0");
uc.DefineVariable("total_response_time = 0.0");
uc.DefineVariable("max_response_time = 0.0");


Step 2: Defining the Log Parsing Rule

Next, we create our Transformer and define the main rule. The pattern is designed to be robust against variable whitespace. The key is using {@String:request} to capture the entire HTTP request, including the spaces inside it, as a single token.

csharp
var t = new uCalc.Transformer();

// The pattern captures all fields from a valid log line.
var pattern = "{@date} {@time} {@level} {@ip} {@String:request} {@Number:status} {@Number:time}ms";


Step 3: The Aggregation Logic

The replacement string is where the magic happens. It contains no visible text, only a sequence of {@Exec} calls that update our metric variables for each match. Note that we must use Double(time) to convert the captured time string into a number for calculations.

csharp
var replacement = ""
+ "{@Exec: request_count++}"
+ "{@Exec: total_response_time = total_response_time + Double(time)}"
+ "{@Exec: max_response_time = Max(max_response_time, Double(time))}"
+ "{@Exec: if(status >= 400, error_count++)}";

t.FromTo(pattern, replacement);


Step 4: Running the Analysis and Displaying Results

Finally, we pass the entire multi-line log string to the Transform method. After it completes, our metric variables will hold the final aggregated values, which we can retrieve with EvalStr.

csharp
// After running t.Transform(logText)...

Console.WriteLine("--- Log Analysis Summary ---");
Console.WriteLine($"Total Requests: {uc.EvalStr("request_count")}");
Console.WriteLine($"Total Errors: {uc.EvalStr("error_count")}");
Console.WriteLine($"Average Response Time: {uc.EvalStr("total_response_time / request_count")}ms");
Console.WriteLine($"Maximum Response Time: {uc.EvalStr("max_response_time")}ms");


⚖️ Why uCalc? (Comparative Analysis)

• vs. Regex: Crafting a single, robust regex to capture all these fields while handling variable whitespace would be extremely complex and difficult to maintain. Furthermore, regex has no built-in mechanism for performing calculations during the match.
• vs. Manual Split(): A simple Split(' ') would fail because of the spaces inside the quoted request string (e.g., "GET /api/users HTTP/1.1"). uCalc's tokenizer correctly identifies this as a single string token, making the pattern robust and simple.