PowerShell wget : Awesome Way to Download a File (2023)

Do you need to download files from the web but hate repeatedly clicking links? If your job involves downloading files from the web regularly, you will probably want to automate the task. Why not use PowerShell to download files much like an alternative PowerShell wget?

Not a reader? Watch this related video tutorial!

Not seeing the video? Make sure your ad blocker is disabled.

Windows PowerShell and PowerShell comes with file-download capabilities. Using PowerShell to download files is a matter of knowing which cmdlets and .NET classes to use and how to use them.

In this article, you’ll learn the various ways to use PowerShell to download files from the web.

Table of Contents

Prerequisites

Run a free, read-only audit of your Active Directory for password vulnerabilities like breached password use and expired accounts. See how your IT security stacks up with a scan fromSpecops Password Auditor.

Since this is a learning-by-doing article, there are some prerequisites to ensure that you can follow the examples. Below are the basic requirements.

  • A computer that is running on Windows 10 or higher. This computer is where you will run the scripts/commands featured in this article.
  • Windows PowerShell 5.1 or PowerShell 7.1 (recommended).
    • Windows 10 already includes Windows PowerShell 5.1.
  • A web site that hosts the files to download.
    • For non-authenticated file downloads, consider using the Tele2 Speedtest site, which is free.
    • If you want to test file downloads with authorization, you may have to build your HTTP file server. An example of a free HTTP file server is HFS by Rejetto.

Using PowerShell to Download Files from URLs: Four Ways

There are four methods to use PowerShell to download files that do not depend on third-party tools. These are:

  • Invoke-WebRequest
  • Invoke-RestMethod
  • Start-BitsTransfer
  • .NET WebClient Class.

Whichever one of these four methods you use, the logic and components to make them work are the same. There must be a source URL pointing to the file’s location and the destination path to save the downloaded files. If required by the webserver, you need to enter the credentials as well.

The next sections show each of these four methods. In the end, it’s up to you to decide which way you would adapt when using PowerShell to download files.

Using Invoke-WebRequest as a PowerShell wget Alternative

The first method in PowerShell to download files is by using the Invoke-WebRequest cmdlet. Perhaps the most used cmdlet in this article, Invoke-WebRequest, can download HTTP, HTTPS, and FTP links.

Whether the source location requires users to log in, the Invoke-WebRequest cmdlet can handle requests with credentials as well.

To download a file, the syntax below shows the minimum parameters required to achieve the desired outcome.

Invoke-WebRequest -Uri <source> -OutFile <destination>
(Video) How To Use wget To Download a File (and a full website)

For example, the code below downloads a file with the name 10MB.zip from a website. Then it saves the downloaded file to C:\dload\10MB.zip. You may copy the code below and paste it into your PowerShell session to test.

# Source file location$source = 'http://speedtest.tele2.net/10MB.zip'# Destination to save the file$destination = 'c:\dload\10MB.zip'#Download the fileInvoke-WebRequest -Uri $source -OutFile $destination

The demonstration below shows the expected result after running the code above in PowerShell. As you can see, the file download was successful.

PowerShell wget : Awesome Way to Download a File (1)

How about if the source requires authentication before allowing access? For example, the code below downloads a file from a private website where users must log in.

$source = 'https://mirror.lzex.ml/100MB.zip'$destination = 'c:\dload\100MB.zip'Invoke-WebRequest -Uri $source -OutFile $destination

However, the download failed due to unauthorized access.

PowerShell wget : Awesome Way to Download a File (2)

If authentication is required, you should add a credential to the request using the -Credential parameter. The first line in the code below prompts you to enter the credential (username and password) and stores it to the $credential variable.

$credential = Get-Credential$source = 'https://mirror.lzex.ml/100MB.zip'$destination = 'c:\dload\100MB.zip'Invoke-WebRequest -Uri $source -OutFile $destination -Credential $credential

The demonstration below shows what you’d expect to see when you run the above code in PowerShell. As you can see, the Get-Credential cmdlet prompted a PowerShell credential request. This time, using the credential with Invoke-WebRequest resulted in a successful download.

PowerShell wget : Awesome Way to Download a File (3)

Related: Using the PowerShell Get-Credential Cmdlet and all things credentials

Looking Out for Parsing Errors when using Invoke-WebRequest

A crucial thing to remember when using Invoke-WebRequest in Windows PowerShell is that, by default, this cmdlet uses the Internet Explorer engine to parse data. The error below may happen when using Invoke-WebRequest on computers without the Internet Explorer in it.

You’ll have to re-issue your command, but this time, include the -UseBasicParsing switch.

Invoke-WebRequest -Uri <source> -OutFile <destination> -UseBasicParsing

In Windows PowerShell, you may receive an error message: The response content cannot be parsed because the Internet Explorer engine is not available, or Internet Explorer’s first-launch configuration is not complete. Specify the UseBasicParsing parameter and try again.

(Video) How To Download Files From PowerShell In Windows 10

Starting with PowerShell Core 6.0, the Invoke-WebRequest cmdlet uses basic parsing only. As such, the -UseBasicParsing parameter is no longer necessary.

Using Invoke-RestMethod

The Invoke-RestMethod cmdlet is more about sending an HTTP or HTTPS request to a RESTful web service. This cmdlet is more suited for requests that interact with REST APIs such as Microsoft Graph API.

When it comes to downloading files straight from the web, Invoke-RestMethod is an excellent contender. Do not be deceived into thinking otherwise. There is not much difference between using Invoke-RestMethod and Invoke-WebRequest when used for downloading files from a direct web link.

Downloading a File using Invoke-RestMethod

To download a file using Invoke-RestMethod, use the syntax below. You’ll notice that the command uses the same parameters as Invoke-WebRequest.

Invoke-RestMethod -Uri <source> -OutFile <destination>

In the example code below, the file is downloaded from the URL value in the $source variable. Then, saved to the path defined in the $destination variable.

$source = 'http://speedtest.tele2.net/10MB.zip'$destination = 'c:\dload\10MB.zip'Invoke-RestMethod -Uri $source -OutFile $destination

If the source requires authentication, you can pass the credentials using the -Credential parameter. The example below prompts for the credentials and stores it to the $credential variable. The value of the $credential variable is then passed to the -Credential parameter.

Also, since the file link is an HTTP source and not HTTPS, it means that you are sending an unencrypted authentication. Typically, you should avoid using HTTP sources for security. But if you must use an HTTP source, you need to add the -AllowUnencryptedAuthentication switch to your command.

$credential = Get-Credential$source = 'http://speedtest.tele2.net/10MB.zip'$destination = 'c:\dload\10MB.zip'Invoke-RestMethod -Uri $source -OutFile $destination -Credential $credential -AllowUnencryptedAuthentication

Using Start-BitsTransfer

Start-BitsTransfer is designed specifically for transferring files between client and server computers. This PowerShell cmdlet is dependent on the Background Intelligent Transfer Service (BITS) that is native to the Windows operating system.

Because Start-BitsTransfer requires BITS to work means that this cmdlet is not available on non-Windows computers. On the flipside, Start-BitsTransfer enjoys the benefits of BITS itself. Some of these benefits are:

  • Network bandwidth and usage awareness.
  • Interruption handling (resume, auto-resume, pause, etc.)
  • Downloading multiple files as background jobs.
  • Ability to set download job priorities.

Downloading a File

The fundamental way to use Start-BitsTransfer in PowerShell to download a file is to specify a source and destination. Using the script below, you only need to change the $source and $destination values according to your requirements.

$source = 'http://speedtest.tele2.net/100MB.zip'$destination = 'c:\dload\100MB.zip'Start-BitsTransfer -Source $source -Destination $destination

As you can see from the demo below, the file is downloaded to the path c:\dload\100MB.zip.

PowerShell wget : Awesome Way to Download a File (4)

Suppose the destination is not specified, Start-BitsTransfer downloads and saves the file to the current working directory. For example, if you run Start-BitsTransfer from C:\dload, the file downloads to the same directory.

(Video) Powershell: How To Download A File

For downloads that require authentication, Start-BitsTransfer has a -Credential parameter that accepts a PSCredential object.

Downloading Multiple Files

To demonstrate downloading multiple files, you’ll need to create a CSV file with two columns. Name the file filelist.txt. The first column should contain the link to the source, while the second column must contain the destination path. The file contents would like the one below.

# source,destinationhttp://speedtest.tele2.net/1MB.zip,c:\dload\1MB.ziphttp://speedtest.tele2.net/10MB.zip,c:\dload\10MB.ziphttp://speedtest.tele2.net/100MB.zip,c:\dload\100MB.zip

Related: Managing CSV Files in PowerShell with Import-Csv

Once the CSV file is ready, use the command below to begin the file download. The command imports the CSV file using Import-Csv and passes the contents to Start-BitsTransfer.

Import-Csv .\filelist.csv | Start-BitsTransfer

Refer to the demo below to see how the code above works. As you can see, the download starts, and you see the download progress. The PowerShell prompt is not available during the download process.

PowerShell wget : Awesome Way to Download a File (5)

Suppose you want to start the download process as a background job. To do so, you only have to add the -Asynchronous switch at the end of the Start-BitsTransfer command.

Import-Csv .\filelist.csv | Start-BitsTransfer -Asynchronous

Initially, the state of each job would show connecting. The screenshot below shows each file download’s job id.

PowerShell wget : Awesome Way to Download a File (6)

Now that you’ve started the download process, you’ll want to check whether the download has been completed. To check the download job status, use the Get-BitsTransfer cmdlet. As you can see below, the download jobs’ status has changed to Transferred.

PowerShell wget : Awesome Way to Download a File (7)

Using WebClient Class and HttpClient Class (.NET Framework)

PowerShell is based on .NET, and its nature makes it capable of leveraging the power of .NET itself. There’s two .NET class you can use in PowerShell to download files; WebClient and HttpClient.

If you want to know more about these two .NET class in more development and technical way, you could start with → When to use WebClient vs. HttpClient vs. HttpWebRequest. In the next section, you will learn how to use WebClient and HttpClient in PowerShell to download files from the web.

Downloading a File using System.Net.WebClient

To use the WebClient class, you need to initiate an object as a System.Net.WebClient **type. In the example below, the $webClient is the new System.Net.WebClient object. Then, using the DownloadFile() method starts the download of the file from the source.

(Video) how to download files with powershell

Related: Using PowerShell Data Types Accelerators to Speed up Coding

Please copy the code below and run it in your PowerShell session to test. Note that you will not see any progress or output on the screen unless there’s an error. However, the PowerShell prompt will be locked until the download is complete.

# Define the source link and destination path$source = 'http://speedtest.tele2.net/10MB.zip'$destination = 'c:\dload\10MB.zip'# Create the new WebClient$webClient = [System.Net.WebClient]::new()# Download the file$webClient.DownloadFile($source, $destination)

If the source requires authentication to allow the file download, you can use the code below. The first line prompts for the credential and stores it to the $credentials variable. The value of $credential is then included in the file download request.

# Prompt for username and password$credentials = Get-Credential$source = 'http://speedtest.tele2.net/10MB.zip'$destination = 'c:\dload\10MB.zip'# Create the new WebClient$webClient = [System.Net.WebClient]::new()# Add the credential$webClient.Credentials = $credentials# Download the file$webClient.DownloadFile($source, $destination)

According to this Microsoft document: “We don’t recommend that you use the WebClient class for new development. Instead, use the System.Net.Http.HttpClient class.”

It appears that the WebClient class is obsolete, and the new class that Microsoft is endorsing is the HttpClient class. Don’t worry, though. The next section talks about using the HttpClient class in PowerShell to download files from the web.

Downloading a File using System.Net.Http.HttpClient

Like the WebClient class, you need to create first the System.Net.Http.HttpClient. Using the code below downloads the file from the $source to the $destination. Refer to the comments above each line to know what each line of code does.

The code below is live, and you can test it by running it in your PowerShell session.

# Set the source and destination$source = 'http://speedtest.tele2.net/10MB.zip'$destination = 'c:\dload\10MB.zip' # Create the HTTP client download request$httpClient = New-Object System.Net.Http.HttpClient$response = $httpClient.GetAsync($source)$response.Wait() # Create a file stream to pointed to the output file destination$outputFileStream = [System.IO.FileStream]::new($destination, [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write) # Stream the download to the destination file stream$downloadTask = $response.Result.Content.CopyToAsync($outputFileStream)$downloadTask.Wait() # Close the file stream$outputFileStream.Close()

In situations where downloading a file requires authentication, you need to add the credential to the HttpClient object. To include a credential to the file download request, create a new System.Net.Http.HttpClientHandler object to store the credentials.

You can copy the code below and run it in PowerShell to test. Or you can also run it as a PowerShell script. In this example, the code is saved as download-file.ps1.

# Set the source and destination$source = 'http://speedtest.tele2.net/10MB.zip'$destination = 'c:\dload\10MB.zip' # Prompt for credentials$credentials = Get-Credential# Create the HTTP client download request with credentials$handler = New-Object System.Net.Http.HttpClientHandler$handler.Credentials = $credentials$httpClient = New-Object System.Net.Http.HttpClient($handler)$response = $httpClient.GetAsync($source)$response.Wait() # Create a file stream to pointed to the output file destination$outputFileStream = [System.IO.FileStream]::new($destination, [System.IO.FileMode]::Create, [System.IO.FileAccess]::Write) # Stream the download to the destination file stream$downloadTask = $response.Result.Content.CopyToAsync($outputFileStream)$downloadTask.Wait() # Close the file stream$outputFileStream.Close()

The demo below shows the result when running the PowerShell script to download the file.

At the start, the directory only has the script file in it. There’s a prompt to enter the username and password. Then, the script proceeds to download the file. After downloading the file, you can see that the new file is now inside the destination directory.

PowerShell wget : Awesome Way to Download a File (8)

Conclusion

Run a free, read-only audit of your Active Directory for password vulnerabilities like breached password use and expired accounts. See how your IT security stacks up with a scan fromSpecops Password Auditor.

(Video) PowerShell For DevOps - Download File Using PowerShell

Windows PowerShell and PowerShell Core come with built-in capabilities to download files, acting as a PowerShell wget alternative! Whether downloading password-protected sources, single or multiple files – a PowerShell way is available to you.

The file download methods covered in this article works on both Windows PowerShell and PowerShell Core. This means that these methods apply to both Windows and Non-Windows systems, with the exclusion of Start-BitsTransfer.

And since PowerShell is more than a command prompt, you can translate what you learned into scripts. For you, that would mean an opportunity for automation. No more copying URLs, clicking links, and waiting for downloads manually.

FAQs

Does wget work in PowerShell? ›

PowerShell provides several useful commands known as cmdlets; Invoke-WebRequest is one of them. You can run wget from PowerShell, which is an alias for the Invoke-WebRequest command.

Can PowerShell download files from a URL? ›

PowerShell can download files from the Internet and your local network to your computer. Learn how to use PowerShell's Invoke-WebRequest and Start-BitsTransfer cmdlets to download files here.

How do I run wget in PowerShell? ›

While not to be confused with the GNU version of Wget, which is also available as an installable application for Windows clients, the Invoke-WebRequest cmdlet may be called by simply launching PowerShell, typing in wget, and pressing the Enter key.

How does wget command work? ›

Wget is a networking command-line tool that lets you download files and interact with REST APIs. It supports the HTTP , HTTPS , FTP , and FTPS internet protocols. Wget can deal with unstable and slow network connections. In the event of a download failure, Wget keeps trying until the entire file has been retrieved.

Can I use curl in PowerShell? ›

curl in PowerShell uses Invoke-WebRequest . From PowerShell 3.0 and above, you can use Invoke-WebRequest , which is equivalent to curl .

What's the difference between curl and wget? ›

Wget requires no extra options to simply download a remote URL to a local file, while curl requires -o or -O. Wget supports only GnuTLS or OpenSSL for SSL/TLS support. Wget supports only Basic auth as the only auth type over HTTP proxy.

How do I use wget on Windows? ›

WGET is a free tool to crawl websites and download files via the command line.
...
Download Wget on Windows
  1. Download wget for Windows and install the package.
  2. Copy the wget.exe file into your C:\Windows\System32 folder.
  3. Open the command prompt (cmd.exe) and run wget to see if it is installed.
Feb 21, 2021

How do I download a directory in PowerShell? ›

Powershell script to Download Specific folders from Sharepoint
  1. #Function to Download All Files from a SharePoint Online Folder - Recursively.
  2. Function Download-SPOFolder([Microsoft. ...
  3. {
  4. #Get the Folder's Site Relative URL.
  5. $FolderURL = $Folder. ...
  6. $LocalFolder = $DestinationFolder + ($FolderURL -replace "/","\")
Mar 25, 2022

How do I download a file from SharePoint using PowerShell? ›

To download all files from a SharePoint Online document library, browse to your SharePoint document library >> Select all Files and folders. Click on the “Download” button from the toolbar. You can use the “File Explorer view” or PowerShell as alternative options to download the document library.

How do I download a ZIP file from PowerShell? ›

PowerShell Download Zip File from Url
  1. Assign Download zip file from url path to variable.
  2. Get file name along with extension from url using Split-Path cmdlet.
  3. Set destination folder path for unzip files.
  4. Use Invoke-WebRequest cmdlet for above url to save response body and create shell object.

How do I run a PowerShell script from the command line? ›

Running a PowerShell script from the Command Prompt

If you would like to run a PowerShell script in CMD, you'll need to execute it by calling the PowerShell process with the -File parameter, as shown below: PowerShell -File C:\TEMP\MyNotepadScript. ps1. PowerShell -File C:\TEMP\MyNotepadScript.

How do I download from the Windows command prompt? ›

Download a file from the command line in Windows
  1. wget http://example.org/picture.jpg. Straight to the point. ...
  2. curl http://example.org/picture.jpg -O picture.jpg. Easy right? ...
  3. Invoke-WebRequest http://example.org/picture.jpg -O picture.jpg. Either way, now you know how to download a file from the command line.
Jul 25, 2017

How do I copy a file in PowerShell? ›

Copy-Item cmdlet is used to copy a file by passing the path of the file to be copied and destination path where the file is to be copied.

How do I download a website using wget? ›

To download a single HTML page (or a handful of them, all specified on the command-line or in a -i URL input file) and its (or their) requisites, simply leave off -r and -l: wget -p http://<site>/1.html Note that Wget will behave as if -r had been specified, but only that single page and its requisites will be ...

What can I use instead of wget? ›

The best alternative is aria2, which is both free and Open Source. Other great apps like Wget are cURL, uGet, ArchiveBox and HTTPie for Terminal. Wget alternatives are mainly Download Managers but may also be Website Downloaders or HTTP Clients.

How do I run a wget script? ›

To use the wget script:
  1. Right-click on the link to the wget script at the bottom of the download page, and save it to your local machine. ...
  2. Make sure the wget file is executable. ...
  3. Run the script by typing "./wget_NNNN.sh" from inside the same directory as the script.

How do you wget a file to a specific directory? ›

Wget – Download files to Specific Directory
  1. Using wget -O Option. Use -O or --output-document=FILE option will truncate FILE immediately, and all downloaded content will be written to FILE. ...
  2. Using wget -P Option. Alternatively, Use -P or --directory-prefix=PREFIX . ...
  3. Conclusion.
Jun 6, 2021

How do I run a curl exe in PowerShell? ›

The conclusion is that if you need to use the curl (as same as in the Windows command prompt) in PowerShell then you need to call the curl executable( curl.exe ) directly. Else, you should stick to the PowerShell curl alias which resolves to the Invoke-WebRequest cmdlet under the hood.

What is curl used for? ›

curl (short for "Client URL") is a command line tool that enables data transfer over various network protocols. It communicates with a web or application server by specifying a relevant URL and the data that need to be sent or received. curl is powered by libcurl, a portable client-side URL transfer library.

Is there a curl equivalent in Windows? ›

What are the alternatives to Curl on Windows? Wget is a free command-line tool for downloading files using the HTTP, HTTPS, and FTP protocols. Wget is a non-interactive command-line tool that can be easily invoked from scripts, deferred commands, non-X-Windows terminals, etc.

How do I download an Excel file from SharePoint using PowerShell? ›

To download all files from a SharePoint Online document library, browse to your SharePoint document library >> Select all Files and folders. Click on the “Download” button from the toolbar. You can use the “File Explorer view” or PowerShell as alternative options to download the document library.

How do I download a directory in PowerShell? ›

Powershell script to Download Specific folders from Sharepoint
  1. #Function to Download All Files from a SharePoint Online Folder - Recursively.
  2. Function Download-SPOFolder([Microsoft. ...
  3. {
  4. #Get the Folder's Site Relative URL.
  5. $FolderURL = $Folder. ...
  6. $LocalFolder = $DestinationFolder + ($FolderURL -replace "/","\")
Mar 25, 2022

How do I copy a file in PowerShell? ›

Copy-Item cmdlet is used to copy a file by passing the path of the file to be copied and destination path where the file is to be copied.

How do I run a PowerShell script from the command line? ›

Running a PowerShell script from the Command Prompt

If you would like to run a PowerShell script in CMD, you'll need to execute it by calling the PowerShell process with the -File parameter, as shown below: PowerShell -File C:\TEMP\MyNotepadScript. ps1. PowerShell -File C:\TEMP\MyNotepadScript.

Videos

1. How To Download Bulk Images From URLs in 5 Minutes - PowerShell (No Softwere's Required) 100% Works
(Solutions First)
2. Download Files/HTML from the web in Powershell!
(MrPowerScripts)
3. Windows 11 - Download files via Powershell/Terminal
(Hidde Tieleman)
4. How to download Wget and download full website with cmd - Wget Tutorial Windows
(Orbit Brain)
5. Powershell For Hackers - Invoke WebRequest
(I am Jakoby)
6. PowerShell for Pentesters - Download Cradles
(Joe Helle)
Top Articles
Latest Posts
Article information

Author: Manual Maggio

Last Updated: 10/19/2022

Views: 6727

Rating: 4.9 / 5 (69 voted)

Reviews: 92% of readers found this page helpful

Author information

Name: Manual Maggio

Birthday: 1998-01-20

Address: 359 Kelvin Stream, Lake Eldonview, MT 33517-1242

Phone: +577037762465

Job: Product Hospitality Supervisor

Hobby: Gardening, Web surfing, Video gaming, Amateur radio, Flag Football, Reading, Table tennis

Introduction: My name is Manual Maggio, I am a thankful, tender, adventurous, delightful, fantastic, proud, graceful person who loves writing and wants to share my knowledge and understanding with you.