You want to get the content of a file.

Provide the filename as an argument to the Get-Content cmdlet:

PS > $content = Get-Content c:\temp\file.txt

Place the filename in a ${ } section to use the cmdlet Get-Content variable syntax:

PS > $content = ${c:	empile.txt}

Provide the filename as an argument to the ReadAllText() method to use the System.IO.File class from the .NET Framework:

PS > $content = [System.IO.File]::ReadAllText("c:\temp\file.txt")

PowerShell offers three primary ways to get the content of a file. The first is the Get-Content cmdlet—the cmdlet designed for this purpose. In fact, the Get-Content cmdlet works on any PowerShell drive that supports the concept of items with content. This includes Alias:, Function:, and more. The second and third ways are the Get-Content variable syntax, and the ReadAllText() method.

When working against files, the Get-Content cmdlet returns the content of the file line-by-line. When it does this, PowerShell supplies additional information about that output line. This information, which PowerShell attaches as properties to each output line, includes the drive and path from where that line originated, among other things.

While useful, having PowerShell attach this extra information when you are not using it can sometimes slow down scripts that operate on large files. If you need to process a large file more quickly, the Get-Content cmdlet's ReadCount parameter lets you control how many lines PowerShell reads from the file at once. With a ReadCount of 1 (which is the default), PowerShell returns each line one-by-one. With a ReadCount of 2, PowerShell returns two lines at a time. With a ReadCount of less than 1, PowerShell returns all lines from the file at once.

If performance is a primary concern, the [File]::ReadAllText() method from the .NET Framework reads a file most quickly from the disk. Unlike the Get-Content cmdlet, it does not split the file into newlines, attach any additional information, or work against any other PowerShell drives. Like the Get-Content cmdlet with a ReadCount of less than 1, it reads all the content from the file before it returns it to you—so be cautious when using it on extremely large files.

For more information about the Get-Content cmdlet, type Get-Help Get-Content. For information on how to work with more structured files (such as XML and CSV), see Chapter 10, Structured Files. For more information on how to work with binary files, see the section called “Parse and Manage Binary Files”.

You want to find a string or regular expression in a file.

To search a file for an exact (but case insensitive) match, use the –Simple parameter of the Select-String cmdlet:

PS > Select-String -Simple SearchText file.txt

To search a file for a regular expression, provide that pattern to the Select-String cmdlet:

PS > Select-String "\(...\) ...-...." phone.txt

To Recursively search all *.txt files for a regular expression, pipe the results of Get-ChildItem to the Select-String cmdlet:

PS > Get-ChildItem -Filter *.txt -Recurse | Select-String pattern

The Select-String cmdlet is the easiest way to search files for a pattern or specific string. In contrast to the traditional text-matching utilities (such as grep) that support the same type of functionality, the matches returned by the Select-String cmdlet include detailed information about the match itself.

PS > $matches = Select-String "output file" transcript.txt
PS > $matches | Select LineNumber,Line

                        LineNumber Line
                        ---------- ----
                                 7 Transcript started, output file...

With a regular expression match, you'll often want to find out exactly what text was matched by the regular expression. PowerShell captures this in the Matches property of the result. For each match, the Value property represents the text matched by your pattern.

PS > Select-String "\(...\) ...-...." phone.txt | Select -Expand Matches

...
Value    : (425) 555-1212

...
Value    : (416) 556-1213

If your regular expression defines groups (parenthesis), you can access the text matched by those groups through the Groups property. The first group (Group[0]) represents all of the text matched by your pattern. Additional groups (1 and on) represent the groups you defined. In this case, we add additional parenthesis around the area code to capture it.

PS > Select-String "\((...)\) ...-...." phone.txt |
>>     Select -Expand Matches | Foreach { $_.Groups[1] }
>>


Success  : True
Captures : {425}
Index    : 1
Length   : 3
Value    : 425

Success  : True
Captures : {416}
Index    : 1
Length   : 3
Value    : 416

If your regular expression defines a named capture (with the text ?<Name> at the beginning of a group), the Groups collection lets you access those by name. In this example, we capture the area code using AreaCode as the capture name.

PS > Select-String "\((?<AreaCode>...)\) ...-...." phone.txt |
>>      Select -Expand Matches | Foreach { $_.Groups["AreaCode"] }
>>


Success  : True
Captures : {425}
Index    : 1
Length   : 3
Value    : 425

Success  : True
Captures : {416}
Index    : 1
Length   : 3
Value    : 416

By default, the Select-String cmdlet captures only the first match per line of input. If the input can have multiple matches per line, use the -AllMatches parameter.

PS > Get-Content phone.txt
(425) 555-1212
(416) 556-1213 (416) 557-1214

PS > Select-String "\((...)\) ...-...." phone.txt |
>>     Select -Expand Matches | Select -Expand Value
>>
(425) 555-1212
(416) 556-1213

PS > Select-String "\((...)\) ...-...." phone.txt -AllMatches |
>>     Select -Expand Matches | Select -Expand Value
>>
(425) 555-1212
(416) 556-1213
(416) 557-1214

For more information about captures, named captures, and other aspects of Regular Expressions, see Appendix B, Regular Expression Reference.

If you want to search multiple files of a specific extension, the Select-String cmdlet lets you use wildcards (such as *.txt) on the filename. For more complicated lists of files (which includes searching all files in the directory), it is usually more useful to use the Get-ChildItem cmdlet to generate the list of files as shown previously.

Since the Select-String cmdlet outputs the filename, line number, and matching line for every match it finds, this output may sometimes be too much detail. A perfect example is when you are searching for a binary file contains a specific string. Binary files rarely make sense when displayed as text, so your screen quickly fills with apparent garbage.

The solution to this problem comes from Select-String's –Quiet switch. It simply returns True or False, depending on whether the file contains the string. So, to find the DLL in the current directory that contains the text "Debug":

Get-ChildItem | Where { $_ | Select-String "Debug" -Quiet }

Two other common tools used to search files for text are the –match operator and the switch statement with the –file option. For more information about those, see the section called “Search a String for Text or a Pattern” and the section called “Manage Large Conditional Statements with Switches”. For more information about the Select-String cmdlet, type Get-Help Select-String.

You want to parse and analyze a text-based logfile using PowerShell's standard object management commands.

Use the Convert-TextObject script given in the section called “Program: Convert Text Streams to Objects” to work with text-based logfiles. With your assistance, it converts steams of text into streams of objects, which you can then easily work with using PowerShell's standard commands.

The Convert-TextObject script primarily takes two arguments:

As an example, you can use patch logs from the Windows directory. These logs track the patch installation details from updates applied to the machine (except for Windows Vista). One detail included in these logfiles are the names and versions of the files modified by that specific patch, as shown in Example 9.1, “Getting a list of files modified by hotfixes”.

PS > cd $env:WINDIR
PS > $parseExpression = "(.*): Destination:(.*) \((.*)\)"
PS > $files = dir kb*.log -Exclude *uninst.log
PS > $logContent = $files | Get-Content | Select-String $parseExpression
PS > $logContent

(...)
0.734: Destination:C:\WINNT\system32\shell32.dll (6.0.3790.205)
0.734: Destination:C:\WINNT\system32\wininet.dll (6.0.3790.218)
0.734: Destination:C:\WINNT\system32\urlmon.dll (6.0.3790.218)
0.734: Destination:C:\WINNT\system32\shlwapi.dll (6.0.3790.212)
0.734: Destination:C:\WINNT\system32\shdocvw.dll (6.0.3790.214)
0.734: Destination:C:\WINNT\system32\digest.dll (6.0.3790.0)
0.734: Destination:C:\WINNT\system32\browseui.dll (6.0.3790.218)
(...)

Like most logfiles, the format of the text is very regular but hard to manage. In this example, you have:

You don't care about any of the text, but the time, file, and file version are useful properties to track:

$properties = "Time","File","FileVersion"

So now, you use the Convert-TextObject script to convert the text output into a stream of objects:

PS > $logObjects = $logContent |
>> Convert-TextObject -ParseExpression $parseExpression -PropertyName $properties
>>

We can now easily query those objects using PowerShell's built-in commands. For example, you can find the files most commonly affected by patches and service packs, as shown by Example 9.2, “Finding files most commonly affected by hotfixes”.

PS > $logObjects | Group-Object file | Sort-Object -Descending Count |
>> Select-Object Count,Name | Format-Table -Auto
>>

Count Name
----- ----
  152 C:\WINNT\system32\shdocvw.dll
  147 C:\WINNT\system32\shlwapi.dll

  128 C:\WINNT\system32\wininet.dll
  116 C:\WINNT\system32\shell32.dll
   92 C:\WINNT\system32\rpcss.dll
   92 C:\WINNT\system32\olecli32.dll
   92 C:\WINNT\system32\ole32.dll
   84 C:\WINNT\system32\urlmon.dll
(...)

Using this technique, you can work with most text-based logfiles.

In Example 9.2, “Finding files most commonly affected by hotfixes”, you got all the information you needed by splitting the input text into groups of simple strings. The time offset, file, and version information served their purposes as is. In addition to the features used by Example 9.2, “Finding files most commonly affected by hotfixes”, however, the Convert-TextObject script also supports a parameter that lets you control the data types of those properties. If one of the properties should be treated as a number or a DateTime, you may get incorrect results if you work with that property as a string. For more information about this functionality, see the description of the –PropertyType parameter in the Convert-TextObject script.

Although most logfiles have entries designed to fit within a single line, some span multiple lines. When a logfile contains entries that span multiple lines, it includes some sort of special marker to separate log entries from each other. Take, for example:

PS > Get-Content AddressBook.txt
Name: Chrissy
Phone: 555-1212
----
Name: John
Phone: 555-1213

The key to working with this type of logfile comes from two places. The first is the –Delimiter parameter of the Get-Content cmdlet, which makes it split the file based on that delimiter instead of newlines. The second is to write a ParseExpression Regular Expression that ignores the newline characters that remain in each record.

PS > $records = gc AddressBook.txt -Delimiter "----"
PS > $parseExpression = "(?s)Name: (\S*).*Phone: (\S*).*"
PS > $records | Convert-TextObject -ParseExpression $parseExpression

Property1                                                   Property2
---------                                                   ---------
Chrissy                                                     555-1212
John                                                        555-1213

The parse expression in this example uses the single line option (?s) so that the (.*) portion of the regular expression accepts newline characters as well. For more information about these (and other) regular expression options, see Appendix B, Regular Expression Reference.

For extremely large logfiles, handwritten parsing tools may not meet your needs. In those situations, specialized log management tools can prove helpful. One example is Microsoft's free Log Parser (http://www.logparser.com). Another common alternative is to import the log entries to a SQL database, and then perform ad hoc queries on database tables, instead.

You want to work with binary data in a file.

Two main techniques are used when working with binary data in a file. The first is to read the file using the Byte encoding, so that PowerShell does not treat the content as text. The second is to use the BitConverter class to translate these bytes back and forth into numbers that you more commonly care about.

Example 9.3, “Get-Characteristics.ps1” displays the "characteristics" of a Windows executable. The beginning section of any executable (a .DLL, .EXE, and several others) starts with a binary section known as the PE (portable executable) header. Part of this header includes characteristics about that file—such as whether the file is a DLL.

For more information about the PE header format, see http://www.microsoft.com/whdc/system/platform/firmware/PECOFF.mspx.

param([string] $filename = $(throw "Please specify a filename."))

$characteristics = @{}
$characteristics["IMAGE_FILE_RELOCS_STRIPPED"] = 0x0001
$characteristics["IMAGE_FILE_EXECUTABLE_IMAGE"] = 0x0002
$characteristics["IMAGE_FILE_LINE_NUMS_STRIPPED"] = 0x0004
$characteristics["IMAGE_FILE_LOCAL_SYMS_STRIPPED"] = 0x0008
$characteristics["IMAGE_FILE_AGGRESSIVE_WS_TRIM"] = 0x0010
$characteristics["IMAGE_FILE_LARGE_ADDRESS_AWARE"] = 0x0020
$characteristics["RESERVED"] = 0x0040
$characteristics["IMAGE_FILE_BYTES_REVERSED_LO"] = 0x0080
$characteristics["IMAGE_FILE_32BIT_MACHINE"] = 0x0100
$characteristics["IMAGE_FILE_DEBUG_STRIPPED"] = 0x0200
$characteristics["IMAGE_FILE_REMOVABLE_RUN_FROM_SWAP"] = 0x0400
$characteristics["IMAGE_FILE_NET_RUN_FROM_SWAP"] = 0x0800
$characteristics["IMAGE_FILE_SYSTEM"] = 0x1000
$characteristics["IMAGE_FILE_DLL"] = 0x2000
$characteristics["IMAGE_FILE_UP_SYSTEM_ONLY"] = 0x4000
$characteristics["IMAGE_FILE_BYTES_REVERSED_HI"] = 0x8000

$fileBytes = Get-Content $filename -ReadCount 0 -Encoding byte

$signatureOffset = $fileBytes[0x3c]

$signature = [char[]] $fileBytes[$signatureOffset..($signatureOffset + 3)]
if([String]::Join('', $signature) -ne "PE`0`0")
{
    throw "This file does not conform to the PE specification."
}

$coffHeader = $signatureOffset + 4

$characteristicsData = [BitConverter]::ToInt32($fileBytes, $coffHeader + 18)

foreach($key in $characteristics.Keys)
{
    $flag = $characteristics[$key]
    if(($characteristicsData -band $flag) -eq $flag)
    {
        $key
    }
}

      

For most files, this technique is the easiest way to work with binary data. If you actually modify the binary data, then you will also want to use the Byte encoding when you send it back to disk:

$fileBytes | Set-Content modified.exe -Encoding Byte

For extremely large files, though, it may be unacceptably slow to load the entire file into memory when you work with it. If you begin to run against this limit, the solution is to use file management classes from the .NET Framework. These classes include BinaryReader, StreamReader, and others. For more information about working with classes from the .NET Framework, see the section called “Work with .NET Objects”. For more information about running scripts, see the section called “Run Programs, Scripts, and Existing Tools”.

You want to create a file for temporary purposes and want to be sure that the file does not already exist.

Use the [System.IO.Path]::GetTempFilename() method from the .NET Framework to create a temporary file:

$filename = [System.IO.Path]::GetTempFileName()
 (... use the file ...)
Remove-Item -Force $filename

It is common to want to create a file for temporary purposes. For example, you might want to search and replace text inside a file. Doing this to a large file requires a temporary file (see the section called “Search and Replace Text in a File”.) Another example is the temporary file used by the section called “Program: Interactively Filter Lists of Objects”.

Often, people create this temporary file wherever they can think of: in C:\, the script's current location, or any number of other places. Although this may work on the author's system, it rarely works well elsewhere. For example, if the user does not use their Administrator account for day-to-day tasks, your script will not have access to C:\ and will fail.

Another difficulty comes from trying to create a unique name for the temporary file. If your script just hardcodes a name (no matter how many random characters it has), it will fail if you run two copies at the same time. You might even craft a script smart enough to search for a filename that does not exist, create it, and then use it. Unfortunately, this could still break if another copy of your script creates that file after you see that it is missing—but before you actually create the file.

Finally, there are several security vulnerabilities that your script might introduce should it write its temporary files to a location that other users can read or write.

Luckily, the authors of the .NET Framework provided the [System.IO.Path]::GetTempFilename() method to resolve these problems for you. It creates a unique filename in a reliable location in a secure manner. The method returns a filename, which you can then use as you want.

By default, the GetTempFilename() method returns a file with a .tmp extension. For most purposes, the file extension does not matter, and this works well. In the rare instances when you need to create a file with a specific extension, the [System.IO.Path]::ChangeExtension() method lets you change the extension of that temporary file. The following example creates a new temporary file that uses the .cs file extension:

$filename = [System.IO.Path]::GetTempFileName()
$newname = [System.IO.Path]::ChangeExtension($filename, ".cs")
Move-Item $filename $newname
(... use the file ...)
Remove-Item $newname

You want to search for text in a file and replace that text with something new.

To search and replace text in a file, first store the content of the file in a variable, and then store the replaced text back in that file as shown in Example 9.4, “Replacing text in a file”.

PS > $filename = "file.txt"
PS > $match = "source text"
PS > $replacement = "replacement text"
PS >
PS > $content = Get-Content $filename
PS > $content
This is some source text that we want
to replace. One of the things you may need
to be careful about with Source
Text is when it spans multiple lines,
and may have different Source Text
capitalization.
PS >
PS > $content = $content -creplace $match,$replacement
PS > $content
This is some replacement text that we want
to replace. One of the things you may need
to be careful about with Source
Text is when it spans multiple lines,
and may have different Source Text
capitalization.
PS > $content | Set-Content $filename

Using PowerShell to search and replace text in a file (or many files!) is one of the best examples of using a tool to automate a repetitive task. What could literally take months by hand can be shortened to a few minutes (or hours, at most).

Example 9.4, “Replacing text in a file” illustrates what is perhaps the simplest (but actually most common) scenario:

If some of those assumptions don't hold true, then this discussion shows you how to tailor the way you search and replace within this file.

$content | Out-File -Encoding Unicode $filename
$content | Out-File -Encoding OEM $filename

PS > $content = Get-Content names.txt
PS > $content
John Doe
Mary Smith
PS > $content -replace '(.*) (.*)','$2, $1'
Doe, John
Smith, Mary

$filename = Get-Item file.txt
$singleLine = [System.IO.File]::ReadAllText($filename.FullName)
$content = $singleLine -creplace "(?s)Source(\s*)Text",'Replacement$1Text'

$filename = "file.txt"
$temporaryFile = [System.IO.Path]::GetTempFileName()

$match = "source text"
$replacement = "replacement text"

Get-Content $filename |
    Foreach-Object { $_ -creplace $match,$replacement | Add-Content $temporaryFile }

Remove-Item $filename
Move-Item $temporaryFile $filename

Both PowerShell and the .NET Framework do a lot of work to hide you from the complexities of file encodings. The Get-Content cmdlet automatically detects the encoding of a file, and then handles all encoding issues before returning the content to you. When you do need to know the encoding of a file, though, the solution requires a bit of work.

Example 9.6, “Get-FileEncoding.ps1” resolves this by doing the hard work for you. Files with unusual encodings are supposed to (and almost always do) have a byte order mark to identify the encoding. After the byte order mark, they have the actual content. If a file lacks the byte order mark (no matter how the content is encoded), Get-FileEncoding assumes the .NET Framework's default encoding of UTF-7. If the content is not actually encoded as defined by the byte order mark, Get-FileEncoding still outputs the declared encoding.

<#

.SYNOPSIS
Gets the encoding of a file

.EXAMPLE
PS >Get-FileEncoding.ps1 .\UnicodeScript.ps1

BodyName          : unicodeFFFE
EncodingName      : Unicode (Big-Endian)
HeaderName        : unicodeFFFE
WebName           : unicodeFFFE
WindowsCodePage   : 1200
IsBrowserDisplay  : False
IsBrowserSave     : False
IsMailNewsDisplay : False
IsMailNewsSave    : False
IsSingleByte      : False
EncoderFallback   : System.Text.EncoderReplacementFallback
DecoderFallback   : System.Text.DecoderReplacementFallback
IsReadOnly        : True
CodePage          : 1201
  
#>

param($file)

$encodings = @{}

$encodingMembers = [System.Text.Encoding] |
    Get-Member -Static -MemberType Property

$encodingMembers | Foreach-Object {
    $encodingBytes = [System.Text.Encoding]::($_.Name).GetPreamble() -join '-'
    $encodings[$encodingBytes] = $_.Name
}

$encodingLengths = $encodings.Keys | Where-Object { $_ } |
    Foreach-Object { ($_ -split "-").Count }

$result = "UTF7"

foreach($encodingLength in $encodingLengths | Sort -Descending)
{
    $bytes = (Get-Content -encoding byte -readcount $encodingLength $file)[0]
    $encoding = $encodings[$bytes -join '-']

    if($encoding)
    {
        $result = $encoding
        break
    }
}

[System.Text.Encoding]::$result 

For more information about running scripts, see the section called “Run Programs, Scripts, and Existing Tools”.

When dealing with binary data, it is often useful to see the value of the actual bytes being used in that binary data. In addition to the value of the data, finding its offset in the file or content is usually important as well.

Example 9.7, “Format-Hex.ps1” enables both scenarios by displaying content in a report that shows both. The leftmost column displays the offset into the content, increasing by 16 bytes at a time. The middle sixteen columns display the hexadecimal representation of the byte at that position in the content. The header of each column shows how far into the 16-byte chunk that character is. The far-right column displays the ASCII representation of the characters in that row.

To determine the position of a byte within the input, add the number at the far-left of the row to the number at the top of the column for that character. For example, 0000230 (shown in the far left) + C (shown at the top of the column) = 000023C. Therefore, the byte in this example is at offset 23C in the content.

<#

.SYNOPSIS
Outputs a file or pipelined input as a hexadecimal display. To determine the
offset of a character in the input, add the number at the far-left of the row
with the the number at the top of the column for that character.

.EXAMPLE
PS >"Hello World" | Format-Hex

            0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F

00000000   48 00 65 00 6C 00 6C 00 6F 00 20 00 57 00 6F 00  H.e.l.l.o. .W.o.
00000010   72 00 6C 00 64 00                                r.l.d.

.EXAMPLE
PS >Format-Hex c:\temp\example.bmp
  
#>

[CmdletBinding(DefaultParameterSetName = "ByPath")]
param(
    [Parameter(ParameterSetName = "ByPath", Position = 0)]
    [string] $Path,
    
    [Parameter(
        ParameterSetName = "ByInput", Position = 0,
        ValueFromPipeline = $true)]
    [Object] $InputObject
)

begin 
{
    [byte[]] $inputBytes = $null
    if($Path) { $inputBytes = [IO.File]::ReadAllBytes( (Resolve-Path $Path) ) }
    
    $counter = 0
    $header = "            0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F"
    $nextLine = "{0}   " -f  [Convert]::ToString(
        $counter, 16).ToUpper().PadLeft(8, '0')
    $asciiEnd = ""

    "`r`n$header`r`n"
}

process
{
    if($InputObject)
    {
        if($InputObject -is [Byte])
        {
            $inputBytes = $InputObject
        }
        else
        {
            $inputString = [string] $InputObject
            $inputBytes = [Text.Encoding]::Unicode.GetBytes($inputString)
        }
    }
    
    foreach($byte in $inputBytes)
    {
        $nextLine += "{0:X2} " -f $byte

        if(($byte -ge 0x20) -and ($byte -le 0xFE))
        {
           $asciiEnd += [char] $byte
        }
        else
        {
           $asciiEnd += "."
        }

        $counter++;

        if(($counter % 16) -eq 0)
        {
            
            "$nextLine $asciiEnd"
            $nextLine = "{0}   " -f [Convert]::ToString(
                $counter, 16).ToUpper().PadLeft(8, '0')
            $asciiEnd = "";
        }
    }
}

end
{
    if(($counter % 16) -ne 0)
    {
       while(($counter % 16) -ne 0)
       {
          $nextLine += "   "
          $asciiEnd += " "
          $counter++;
       }
       "$nextLine $asciiEnd"
    }

    ""
}
 

For more information about running scripts, see the section called “Run Programs, Scripts, and Existing Tools”.