CH8: Windows Memory Forensics

Learning Objectives

By the end of this chapter, students will be able to:

Explain how the Windows operating system manages memory, including the roles of physical RAM, virtual memory, and paging.
Define the Order of Volatility (OOV) and explain why memory must be captured before non-volatile storage.
Identify the challenges of live acquisition, including the risk of data "smearing."
Compare different memory file formats (Raw, Crash Dump, E01) and their compatibility with analysis tools.
Utilize industry-standard tools like FTK Imager, DumpIt, and WinPmem to acquire memory.
Analyze memory dumps using the Volatility 3 Framework to identify rogue processes, network connections, and injected code.
Differentiate between linked-list process monitoring (pslist) and pool tag scanning (psscan) to detect rootkits.
Describe a standard memory forensics investigation workflow from acquisition through reporting.

8.1 Introduction: The Volatile Frontier

In Chapter 7, we explored browser forensics to uncover a suspect's intent and history. However, investigating a powered-off computer misses a crucial dimension of evidence. Modern cybercrime is increasingly "fileless"; sophisticated malware often resides solely in Random Access Memory (RAM), never writing a payload to the disk to avoid detection by traditional antivirus software.

Memory forensics is the art of analyzing the volatile data of a computer. It allows an investigator to see exactly what the computer was "thinking" at the moment of capture: open windows, typed passwords, encryption keys, and active network connections.

Windows Memory Management Fundamentals

To analyze memory, you must first understand how Windows manages it. You might imagine RAM as a simple bucket of data, but the Operating System (OS) uses a complex system called Virtual Memory.

Virtual Addressing: Windows does not give applications direct access to physical hardware. Instead, every process is given a virtual address space (a "Sandbox"). The CPU and the Memory Management Unit (MMU) translate these virtual addresses into physical addresses in the RAM chips.
Paging (pagefile.sys): Physical RAM is expensive and finite. When the OS runs out of physical RAM, it moves the least active data chunks (pages) from RAM to the hard drive. This file acts as an extension of RAM. From a forensic standpoint, the pagefile is a goldmine; it contains artifacts that were once in memory but were swapped out to disk, persisting even after a reboot.
Hibernation (hiberfil.sys): When a laptop enters hibernation, it takes the entire contents of RAM, compresses it, and writes it to hiberfil.sys on the root of the C: drive. This is essentially a snapshot of RAM on disk, allowing investigators to perform memory forensics on a powered-off machine.

The Forensic Treasure Trove: Artifacts in RAM

Before diving into how we capture memory, it is critical to understand what we are capturing. RAM contains the runtime state of the system, offering artifacts that simply do not exist on the hard drive. As an investigator, you can expect to recover the following types of evidence from a memory dump:

Active Processes and Drivers: Unlike a static list of installed programs, RAM reveals exactly what was running at the time of capture. This includes the process hierarchy (Process Tree), which helps identify anomalies like a system service spawning a command shell.
Network Artifacts: One of the most valuable aspects of memory forensics is the visibility of the network stack. You can recover active connections, listening ports, and closed sockets. This allows you to identify "beaconing" malware communicating with a Command and Control (C2) server, even if the connection is encrypted.
Decrypted Keys and Passwords: Many encryption tools (like BitLocker or TrueCrypt) must store their decryption keys in RAM to function. Investigators can extract these "mount keys" from memory to decrypt the hard drive. Similarly, users' passwords or authentication tokens are often temporarily stored in cleartext within memory buffers.
Command Line Arguments: When a user or script launches a program, they often pass arguments (e.g., net use or psexec). These arguments are stored in the process memory and can reveal exactly what the attacker was trying to do, such as the specific IP address they were targeting or the password they typed into the console.
Malicious Code Injection: Advanced malware often "injects" itself into legitimate processes (like notepad.exe or svchost.exe) to hide from security tools. While the hard drive only shows the legitimate notepad.exe file, the memory dump reveals the malicious code running inside it, often marked by suspicious permissions (Read-Write-Execute).
Unsaved Data: Documents, clipboard contents, and chat messages that were open but never saved to disk can often be carved from memory.

8.2 Memory Acquisition

Often you will see recommendations to 'pull the plug' on a system to prevent any programmed anti-forensics processes from running at shutdown. However, if encryption is active (e.g., BitLocker), pulling the plug destroys the decryption keys stored in RAM, rendering the data permanently inaccessible. Therefore, Volatile Data Analysis must occur first.

The Order of Volatility (OOV)

The Order of Volatility dictates the sequence in which evidence should be collected, moving from the most fleeting data to the most permanent.

CPU Cache & Registers: Overwritten with each CPU instruction cycle and lost immediately on power loss. Impossible to capture without specialized hardware.
RAM / Routing Tables / ARP Cache: Disappears immediately upon power loss.
Temporary Files / Swap Space: May persist briefly.
Disk (HDD/SSD): Persists for years.
Archival Media (CD/DVD/Tape): Persists for decades.

The Problem of "Smearing"

Unlike a hard drive, which can be frozen in time, a running computer is constantly changing. As you run a tool to copy RAM, the OS is simultaneously writing new data to RAM. This can result in smearing, where the data changes during the copy process, potentially corrupting the structural integrity of the forensic image. While unavoidable in software-based acquisition, modern tools attempt to minimize this impact by using kernel-level drivers to "lock" pages or copy them at high speeds.

Acquisition Tools

There are several standard tools used to capture memory. Note that running any tool on a live system alters the evidence. For investigations that are to lead to prosecutions, you should utilize lightweight memory capture tools that can be executed from a USB drive, rather than installing software onto the live system. You must always document precisely which tool you ran.

FTK Imager: A staple in the industry. It can capture memory from a live system and save it as a raw dump (.mem) or an AD1 file.
DumpIt: A lightweight command-line utility (often carried on a USB drive) that quickly dumps physical memory to a raw file.
WinPmem: An open-source memory acquisition tool that supports multiple output formats, including raw and the AFF4 forensic container format. WinPmem is actively maintained and widely used in both professional investigations and training environments.
Magnet RAM Capture: A free standalone tool designed to run with minimal footprint, capturing physical RAM with high reliability.

Legal and Chain of Custody Considerations

Memory acquisition is inherently different from imaging a hard drive. You cannot acquire RAM without running software on the live system, which means the act of collection always modifies the evidence to some degree. This reality makes documentation critical for maintaining admissibility in court.

At a minimum, the investigator must record the tool used (including its exact version and hash), the time the acquisition began and ended, the hash of the resulting output file, and who performed the acquisition. This documentation should be included in your formal report and chain of custody log. Without it, a defense attorney can challenge the integrity of the evidence. Memory acquisition does not give you a free pass to skip the fundamentals of evidence handling—if anything, it demands more rigor.

Memory Capture Formats

When performing the acquisition, you will encounter different file formats. The format you choose dictates which analysis tools you can use later.

Raw Memory Dump (.raw, .mem, .dd): This is the "gold standard" for compatibility. A raw dump is a bit-for-bit copy of the physical RAM, containing no headers, metadata, or compression. Because it is unadulterated, it works seamlessly with virtually every analysis tool, most notably the Volatility Framework. The downside is the file size; if the computer has 32GB of RAM, the file will be exactly 32GB.
Windows Crash Dump (.dmp): Windows has a built-in mechanism to save memory during a system crash (Blue Screen of Death). Investigators can sometimes force a system crash (using keyboard commands or specialized tools) to generate a .dmp file. While useful for debugging, these files require specific conversion or valid symbol tables to be read by standard forensic tools.
Hibernation File (hiberfil.sys): As mentioned in section 8.1, this is a compressed file created by the OS. It is not a "capture" format in the traditional sense, but it is a valid source of memory evidence. Volatility 3's layered address space architecture can process hiberfil.sys files natively in many cases. Older tools may require you to decompress and convert the file to a raw format before analysis.
EnCase / Forensic Containers (.E01, .AD1): Commercial tools like FTK Imager or EnCase often wrap the memory dump in an evidence container. These formats are excellent for chain of custody because they include embedded metadata (Case Number, Examiner Name) and integrity hashes. Volatility 3 has improved support for the EWF/E01 format and can often read these files directly. However, raw format remains the most universally compatible option across all tools. If you encounter compatibility issues, converting the image back to raw is a reliable fallback.

8.3 Analysis Frameworks: Volatility

Once you have acquired a raw memory dump (e.g., memdump.raw), you cannot simply open it in a text editor. You need tools that understand the data structures of the Windows OS.

The Volatility Framework (Volatility 3)

Volatility is the industry standard for deep-dive memory forensics. It is a command-line framework written in Python that allows forensic examiners to extract practically any artifact from a memory image.

We will focus on Volatility 3, the modern iteration of the framework. One of the major improvements in Volatility 3 is the elimination of manual profile selection.

Note

Volatility 3 can be downloaded from the Volatility Foundation GitHub at https://github.com/volatilityfoundation/volatility3.

How Volatility 3 Works (Symbol Tables): Instead of forcing the user to guess the operating system version, Volatility 3 uses ISF (Intermediate Symbol Format) files. When you run a plugin against a memory dump, the framework automatically scans the image to identify the kernel version. It then downloads the appropriate symbol data from the Microsoft Symbol Server (or uses a local cache) and generates an ISF file—essentially a map of the OS data structures. This allows for a much faster and more user-friendly workflow compared to previous versions. If you ever encounter a symbol-related error, the ISF terminology will help you troubleshoot the issue.

Volatility 3 Command Syntax

Before we begin investigating processes, let us establish the basic command syntax you will use throughout your labs. Every Volatility 3 command follows the same general pattern:

python3 vol.py -f <path_to_memory_image> <plugin_name>

For example, to list all running processes from a memory dump called memdump.raw:

python3 vol.py -f memdump.raw windows.pslist

Volatility prints its output to the terminal by default. For large datasets, you will want to redirect the output to a text file for easier review:

python3 vol.py -f memdump.raw windows.netscan > netscan_results.txt

Some plugins accept additional arguments. For instance, to dump a specific process by its PID using the windows.pslist plugin:

python3 vol.py -f memdump.raw windows.pslist --pid 1234

Mastering this basic pattern is essential. Once you internalize the syntax, the only thing that changes between commands is the plugin name and its specific options.

A Note on Legacy Volatility 2

In professional environments, you may still encounter the older "Volatility 2." The primary difference you will notice is the requirement for Profiles. In Volatility 2, you must first run a plugin called imageinfo to identify the specific Service Pack and Build number (e.g., Win7SP1x64). You must then manually type this profile into every subsequent command. If you choose the wrong profile, the output will be gibberish. Volatility 3 removes this friction entirely.

8.4 Investigating Processes

The most common goal in memory forensics is identifying malware running as a process.

Know Normal, Find Evil

Before you can identify a malicious process, you must first understand what a healthy Windows system looks like. This concept is often summarized as "Know Normal, Find Evil." If you do not know what legitimate system processes look like, you will either miss real threats or waste hours chasing false positives.

Here are the baseline expectations for critical Windows processes that you should commit to memory:

System (PID 4): Always has PID 4. It has no visible parent process. There should only ever be one instance.
smss.exe (Session Manager): Its parent is System (PID 4). It runs from C:\Windows\System32\. There should only be one persistent instance after boot (child instances start and exit quickly during session creation).
csrss.exe (Client/Server Runtime): There should be exactly two instances—one for Session 0 (system) and one for Session 1 (user). Its parent (smss.exe) will have exited, so the parent PID may appear empty or point to a non-existent process.
wininit.exe: Runs in Session 0. Its parent is smss.exe. There should only be one instance.
winlogon.exe: Runs in Session 1. Its parent is smss.exe. There should only be one instance per interactive user session.
services.exe: Its parent is wininit.exe. There should only be one instance, and it should run from C:\Windows\System32\.
svchost.exe: This is the most commonly impersonated process. Legitimate instances are always children of services.exe and always run from C:\Windows\System32\svchost.exe. There will be many instances, and each should be launched with a -k flag indicating its service group (e.g., -k netsvcs).
lsass.exe (Local Security Authority): Its parent is wininit.exe. There should be exactly one instance. If you see two lsass.exe processes, treat it as a critical finding—attackers frequently target this process to harvest credentials.
explorer.exe: This is the user's desktop shell. Its parent is typically userinit.exe, which will have already exited. There should be one instance per logged-in user.

Any deviation from these baselines—a wrong parent, a wrong file path, an unusual number of instances, or a misspelled name—is a red flag that warrants immediate investigation.

The Process List (`windows.pslist`)

The OS tracks running programs using a doubly linked list. The windows.pslist plugin walks this list, showing every active process, its Process ID (PID), and its Parent Process ID (PPID).

What to look for:

Typosquatting: svch0st.exe vs svchost.exe.
Odd Paths: System binaries running from C:\Temp\ or C:\Users\.
Instance Count: More instances of lsass.exe or services.exe than expected (should be one each).

Finding Hidden Processes (`windows.psscan`)

Advanced malware (Rootkits) can use DKOM (Direct Kernel Object Manipulation) to "unlink" themselves from the active process list. This makes them invisible to Task Manager and pslist.

However, the process data structure still exists in memory; it just isn't in the list. The windows.psscan plugin ignores the list and scans the entire memory range looking for Pool Tags—byte signatures (specifically the Proc tag) that mark the start of an EPROCESS object in kernel memory.

The Discrepancy: If psscan finds a process that pslist does not, you have likely found a rootkit.

The Process Tree (`windows.pstree`)

Malware often creates a distinct "Process Tree" structure. The windows.pstree plugin visualizes the parent-child relationships.

Anomaly: The legitimate chain is wininit.exe → services.exe → svchost.exe. If you see explorer.exe (the user desktop) spawning svchost.exe, or svchost.exe with no parent relationship to services.exe, it is almost certainly malware. Always trace the full lineage, not just the immediate parent.

8.5 Network and Code Artifacts

Network Connections (`windows.netscan`)

Since the capture is of a live system, the network stack is visible. The windows.netscan plugin recovers active connections, listening ports, and closed sockets.

Investigative Value: You can see exactly which IP address a suspect process is communicating with. This is crucial for attribution. Pay close attention to ESTABLISHED connections to external IP addresses from processes that should not be making network calls (e.g., notepad.exe connecting to a foreign IP is a major red flag).

Code Injection (`windows.malfind`)

Malware often injects malicious code into legitimate processes (like notepad.exe) to hide. The windows.malfind plugin scans for memory pages with RWX (Read, Write, Execute) permissions.

Why it matters: Legitimate Windows memory pages are usually Read-Execute (for code) or Read-Write (for data). A page that is both Writable and Executable is highly suspicious, often indicating a hollowed process or shellcode injection. When reviewing malfind output, look for the MZ header (the magic bytes 4D 5A) in the hex dump—this indicates a full Portable Executable (PE) has been injected into the process.

DLL Analysis (`windows.dlllist` and `windows.ldrmodules`)

A Dynamic Link Library (DLL) is a shared code library that multiple programs can use simultaneously. Attackers exploit this architecture through DLL injection—forcing a legitimate process to load a malicious DLL—or DLL side-loading, where a malicious DLL is placed in a directory where a legitimate application will load it instead of the real one.

Your first-pass tool is windows.dlllist. This plugin walks the PEB (Process Environment Block) loader list for each process and displays every DLL that process has loaded, along with the full file path and load address. This is your "what DLLs is this process using?" view. If you see svchost.exe loading a DLL from C:\Users\Public\Downloads\ instead of C:\Windows\System32\, that is an immediate red flag. It is also effective for spotting DLL side-loading, where a legitimate application loads a malicious DLL simply because it was placed in a directory the application searches first.

Your deeper follow-up tool is windows.ldrmodules. This plugin cross-references three internal Windows lists that track loaded DLLs. If a DLL appears in one list but not the others, it may have been injected or deliberately hidden. A DLL that is "unlinked" from the standard loader list but still present in memory is a strong indicator of malicious activity. Because dlllist walks the same loader list that attackers manipulate, it will miss these unlinked DLLs entirely—which is exactly why ldrmodules exists.

The investigative flow is straightforward: run dlllist first to see what is visible, then run ldrmodules to find what has been deliberately concealed.

Handles and Mutexes (`windows.handles`)

Every process interacts with system resources—files, registry keys, network sockets—through handles. The windows.handles plugin enumerates these handles for a given process, providing a detailed picture of what that process is touching.

One particularly useful artifact is the mutex (mutual exclusion object). Malware authors frequently create uniquely named mutexes to ensure only one copy of their malware runs at a time, preventing re-infection. These unique mutex names become Indicators of Compromise (IOCs) that analysts can search for across other systems. If you find a suspicious process, checking its handles for unusual mutex names can confirm malicious intent and help you identify the malware family.

YARA Scanning (`windows.yarascan`)

YARA is an industry-standard tool for creating rules that match byte patterns, strings, and conditions within files or memory. Think of a YARA rule as a custom signature—you define what "bad" looks like, and the scanner finds it.

The windows.yarascan plugin allows you to run YARA rules directly against a memory dump. This is incredibly powerful because it lets you search for known malware signatures, specific strings (like a hardcoded C2 domain), or suspicious byte patterns across the entire memory image. If your organization's threat intelligence team provides YARA rules for a known threat actor, you can use this plugin to determine whether that specific malware was present in memory at the time of capture.

The Command Line (`windows.cmdline`)

Finding the malware process is step one. Finding how it was launched is step two. The windows.cmdline plugin extracts the exact arguments passed to the executable.

Example: You might see nc.exe. Is it malicious? cmdline reveals: nc.exe -l -p 8080 -e cmd.exe. This confirms it is a listener (backdoor) offering a command shell to anyone who connects.

8.6 Practical Investigation Workflow

With all of these plugins at your disposal, it helps to follow a structured workflow rather than running plugins at random. Below is a standard approach for investigating a memory image:

Verify the Image: Run windows.info to confirm the memory dump is recognized and identify the OS version.
Enumerate Processes: Run windows.pslist, windows.psscan, and windows.pstree. Compare the outputs. Look for hidden processes (present in psscan but missing from pslist), suspicious names, wrong parent-child relationships, and unexpected file paths.
Inspect the Network: Run windows.netscan. Identify any active or recently closed connections to external IP addresses. Cross-reference the owning PID with your process list to determine which process initiated the connection.
Hunt for Injection: Run windows.malfind to find memory pages with RWX permissions. Investigate any flagged processes. Run windows.ldrmodules to check for unlinked or hidden DLLs.
Extract Context: For any suspicious process, run windows.cmdline to see how it was launched, windows.handles to see what resources it was accessing, and windows.getsids to determine which user account was running it.
Scan for Known Threats: If you have YARA rules available, run windows.yarascan to check for known malware signatures across the entire image.
Document and Report: Record every command you ran, every finding, and the hashes of your evidence files. Your analysis is only as strong as your documentation.

This workflow is not rigid—a specific case may require you to prioritize network connections over process enumeration, or skip directly to YARA scanning if you have intelligence about a specific threat. However, following a consistent methodology ensures you do not overlook critical evidence.

8.7 Reference: Volatility 3 Plugin Table

Below is a reference table for the most critical Volatility 3 plugins you will use in your labs.

Category	Plugin Name	Description	What to Look For
Info	`windows.info`	Identifies the OS version, architecture, and kernel base.	Used to verify the memory image was successfully recognized.
Processes	`windows.pslist`	Lists running processes by walking the active process links.	Standard process viewing. Look for known malware names and instance counts.
Processes	`windows.psscan`	Scans memory for EPROCESS blocks (pool scanning).	Processes found here but missing in `pslist` are hidden (Rootkits).
Processes	`windows.pstree`	Displays processes in a parent-child tree format.	Anomalies in hierarchy (e.g., `cmd.exe` spawned by `winword.exe`).
Network	`windows.netscan`	Scans for network connections and sockets.	Connections to known C2 (Command & Control) IPs; listening ports on non-servers.
Malware	`windows.malfind`	Scans for injected code/shellcode (VADs with RWX permissions).	Any output here requires immediate investigation. Look for "MZ" headers in the hex dump.
Malware	`windows.dlllist`	Lists DLLs loaded by each process via the PEB loader list.	DLLs loaded from unexpected paths (e.g., `C:\Temp\` or user directories instead of `System32`).
Malware	`windows.ldrmodules`	Detects unlinked DLLs by cross-referencing loader lists.	DLLs that are loaded but hidden from the process loader list.
Malware	`windows.yarascan`	Runs YARA rules against the memory image.	Custom signature matches for known malware families, C2 domains, or suspicious strings.
Registry	`windows.registry.hivelist`	Lists registry hives present in memory.	Locating `NTUSER.DAT` or `SAM` hives for password/hash extraction.
Files	`windows.filescan`	Scans for file objects in memory.	Can find open file handles, helping associate a process with a specific file on disk.
System	`windows.cmdline`	Lists process command-line arguments.	Malicious arguments (e.g., PowerShell encoded commands).
System	`windows.handles`	Enumerates handles (files, registry keys, mutexes) for processes.	Unusual mutex names (IOCs), unexpected file or registry access.
System	`windows.getsids`	Displays the SIDs (Security Identifiers) associated with a process.	Identifying which user account is running a specific malicious process.

8.8 Chapter Summary

Memory forensics changes the game from "what happened?" to "what is happening?". By capturing RAM, we secure the most volatile and often the most valuable evidence available—encryption keys, network connections, and fileless malware. However, the investigator must be careful. The act of observing the memory changes it, and a structured methodology backed by rigorous documentation is required to translate the raw binary of a memory dump into a coherent, defensible story of user activity and system compromise. Knowing what "normal" looks like is just as important as knowing how to use the tools—without that baseline, you are searching for a needle in a haystack without knowing what the needle looks like.

Key Terms Review

Order of Volatility: The sequence of capturing evidence from most fleeting to most permanent.
Smearing: Data corruption caused by the contents of RAM changing while it is being imaged.
DKOM: Direct Kernel Object Manipulation, a technique used by rootkits to hide processes.
Pool Tag: A byte signature in kernel memory that marks the beginning of a specific data structure, such as Proc for process objects.
RWX: Read-Write-Execute permissions, a primary indicator of code injection.
DLL Injection: A technique where malicious code is loaded into a legitimate process via a Dynamic Link Library.
YARA: A pattern-matching language used to create custom signatures for identifying malware in files or memory.
Mutex: A mutual exclusion object used by malware to prevent multiple instances from running; often a unique IOC.
IOC (Indicator of Compromise): A forensic artifact (hash, IP address, mutex name, etc.) that indicates a system has been compromised.
Pagefile: A file on the hard drive that serves as an extension of physical RAM.