Skip to content

BlueDotBrigade/weevil

Repository files navigation

Weevil

Latest Release Latest Build Security Rating

What is Weevil?

WeevilDemo

Weevil is an open-source .NET project that is used by analysts to extract valuable insights from log files. It's all about "boring log files for tasty bytes".

A complete list of features can be found in the release notes.

Key Features

  1. File and Record Level Notes
    • Capture high-level observations as remarks, or low-level details as record comments.
  2. Persisted State
    • Automatically load filter history, record comments, and file level comments when opening a log file.
    • Share the application's state as an XML sidecar with colleagues.
  3. Non-Destructive Operations
    • The Weevil application ensures that the original log file is never modified.
  4. Simplified Callstacks
    • When a record includes an exception call stack, Weevil simplifies the call stack by only displaying business logic references.
  5. Clear Operations
    • This operation removes records from memory, thus reducing the RAM footprint and speeding up the filtering process.

Filtering

One or more filter criteria can be used to show or hide log file records.

  1. Inclusive and Exclusive Filters
    • Display records matching the inclusive filter while hiding those matching the exclusive filter.
  2. Filter Criteria
    1. Plain Text
    2. Regular Expressions
    3. Aliases
      • Frequently used or complex filters can be assigned a unique key that can be used to speed up the filtering process.
      • For example, the #IpAddress key could be assigned to the following filter criteria ^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$.
    4. Monikers
      • Monikers are built-in keys that can be used to query metadata collected by Weevil.
        • For example, the @Comment can be used to identify records that have a user comment.
  3. Multiple Criteria
    • Multiple filter criteria can be combined together using a logical "OR" operator (||).
  4. Pinned Records
    • Pinned records are guaranteed to be included in the filter results.

Navigation

  1. Find
    • Search for text within filtered results.
  2. Go To
    • Jump to specific line numbers or timestamps.
  3. Pinned Records
    • Effortlessly navigate between important records.
  4. Flagged Records
    • Move between records flagged during prior analysis.
  5. Record Comments
    • Navigate between records containing user comments.

Analysis

Utilize Regular expression named groups to identify key data in log files. Leverage Weevil's analysis tools to then extract data and identify trends.

Each analysis tool updates the Comments fields with the values that match the provided named group(s), and the recor's Flagged field is set.

  1. Detect Data
    • For example: extracting URLs from a log file
  2. Detect Data Transitions
    • For example: when a hardware serial number changes
  3. Detect Rising Edges
    • For example: detecting peek CPU usage
  4. Detect Falling Edges
    • For example: detect when a firmware's uptime has reset
  5. Detect Temporal Anomalies
    • For example: detect when records are logged out of order

Furthermore, Weevil includes the ability to generated graphs based on the extracted data.

Extensible Architecture

Maximize potential by developing domain-specific extensions tailored to your business' needs. Weevil can be enhanced by custom plugins:

  1. Log File Parsers
    • Create tailored parsers to accurately interpret log files from various sources and formats, ensuring seamless integration with Weevil.
  2. Log File Analyzers
    • Design specialized analyzers to process and extract valuable insights from the parsed log data, optimizing the analysis for your specific business domain.
  3. Dashboard Insights
    • Develop custom dashboard visualizations and insights that highlight the most relevant information, enabling efficient decision-making and improved understanding of your log data.

Software Development

WPF Application

NuGet Packages

Latest Release NuGet Package
latest version BlueDotBrigade.Weevil.Common.nupkg
latest version BlueDotBrigade.Weevil.Core.nupkg
latest version BlueDotBrigade.Weevil.Windows.nupkg

A .NET application can use Weevil's feature set by directly referencing the BlueDotBrigade.Weevil.Core NuGet package.

For example, one could determine when equipment was changed using the following sample code:

var engine = Engine
   .UsingPath(@"C:\Temp\hardware.log")
   .Open();

// The `UniqueId` regular expression named group is used to
// capture serial hardware serial numbers.
engine.Filter.Apply(
   FilterType.RegularExpression,
   new FilterCriteria(@"Received hardware message. ID=(?<UniqueId>[a-zA-Z0-9]+)"));

// This type of analysis compares the captured serial numbers,
// and flags the record when a value changes.
engine.Analyzer.Analyze(AnalysisType.DetectDataTransition);

foreach (var record in engine.Filter.Results.Where(r => r.Metadata.IsFlagged == true))
{
   Console.WriteLine(
   $"{record.CreatedAt} {record.Metadata.Comment}");
}

Development

Attribute Description
GitHub Latest Release The list of features & bug fixes for the latest Weevil release.
Latest Stable Source code for the most stable version of Weevil.
Latest Code The most up-to-date source code. This branch includes features that are still under development.
Latest Build A value of passing indicates that the main branch is compiling & that the automated tests have passed.
GitHub Repository Size Total size of Weevil's Git repository.
Lines of code Total number of lines of code in the Git repository.
Last Commit Indicates when the Git repository was last updated.
Security Rating SonarCube: Number of security issues detected.
Vulnerabilities SonarCube: Number of security vulnerabilities detected
Maintainability Rating SonarCube: Represents the project's SQALE rating.
Code Smells SonarCube: Characteristics of the code base that suggest the design may have maintenance issues.

Guidelines

  • When working on the WPF application, please be sure to follow the Style Guide for the user interface.

Compiling

The following steps outline how to build Weevil's WPF application:

  1. Download the latest stable release source code.
  2. If you have implemented a custom Weevil plugin:
    • Prior to starting Visual Studio, create the following Windows [environment variable][EnvironmentVariable]:
      • %WEEVIL_PLUGINS_PATH% which refers to the directory where the Weevil plugin assembly (*.dll) can be found.
  3. Using Visual Studio, compile the WPF project: BlueDotBrigade.Weevil.Gui [EnvironmentVariable]: https://en.wikipedia.org/wiki/Environment_variable#Windows

Verification

Software integrity is verified through a number of automated tests which can be found in the /Weevil/Tst/ directory:

  • UnitTests
  • FunctionalTests

Recognition

  • PostSharp
    • PostSharp`s aspect oriented library helps to simplify a code base by reducing boilerplate. Special thanks to the PostSharp team for donating a license.
  • GitHub
    • Free Git repository hosting platform for this project & many others like it.

Open Source Projects

  • Live Charts
    • Beto Rodriguez et al. have developed an impressive WPF charting library. Am looking forward to future releases.
  • Material Design in XAML
    • An excellent WPF library that helps to standardize themes & improve the overall quality of an application's user interface.

Contributors

A special thanks to all of those who have contributed to this project.