当前位置：网站首页>The sharp weapon of web automation -- curl

The sharp weapon of web automation -- curl

2022-07-22 07:37:00 【51CTO】

 cURL Support HTTP、HTTPS、FTP agreement , Also support POST,cookie, Breakpoint continuation ,userAgent,extra header, The speed limit , File size limit , Progress bar, etc .

By default ,linux The distribution of often does not bring cURL, Self installation required .

cURL Usually output the downloaded file to stdout, Output progress information to stderr, If you don't want to display progress information , have access to --silent Options .

Simplest use ：

      
      

       
       $ curl URL --silent
      
      

      
      
    
       
       
  • 1.
    •

Such a command will output the file to the terminal , If you want to write to a file , have access to -O Options .

      
      

       
       $ curl URL --silent -O
      
      

      
      
    
       
       
  • 1.
    •

If the output file name is not specified , Will use from URL File name resolved in , for example 

      
      

       
       $ curl http://linux.web-coding.tech/index.html --silent -O
      
      

      
      
    
       
       
  • 1.
    •

So you get a index.html file .

If you can't get from URL The file name is resolved in , An error will be reported when the command is executed , So you can specify the file name ：

      
      

       
       $ curl http://linux.web-coding.tech/ --silent -o index.html
      
      

      
      
    
       
       
  • 1.
    •

Note that the former is -O, The latter is -o.


curl Support to continue downloading from a specific file offset ,

      
      

       
       $ curl URL -C offset
      
      

      
      
    
       
       
  • 1.
    •

offset Is an integer in bytes .

If you only want to resume at breakpoints , It can be like this 

      
      

       
       $ curl URL -C -URL
      
      

      
      
    
       
       
  • 1.
    •

We can use curl Storage cookie For a file 

      
      

       
       $ curl URL --cookie-jar cookie_file
      
      

      
      
    
       
       
  • 1.
    •

If you want to specify cookie

      
      

       
       $ curl URL --cookie "key1=value1;key2=value2"
      
      

      
      
    
       
       
  • 1.
    •

cookies Yes, it is key=value Given in the form of , Multiple cookie Separated by semicolons .


There are some web pages that will detect visits userAgent, At this time, if you want to access in the specified way , You need to specify userAgent

      
      

       
       $ curl URL --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1"
      
      

      
      
    
       
       
  • 1.
    •

use -A and --user-agent It's the same .


alike , If you need to set the access header

      
      

       
       $ curl URL -H "Host:linux.web-coding.tech" -H "Accept-Language: zh-CN"
      
      

      
      
    
       
       
  • 1.
    •

If you only want to get the header of the request response , have access to -I perhaps -head

      
      

       
       $ curl -I http://linux.web-coding.tech
      
      

      
      
    
       
       
  • 1.
    •


If bandwidth is limited , have access to --limit-rate To limit the download speed 

      
      

       
       $ curl URL --limit-rate 20k
      
      

      
      
    
       
       
  • 1.
    •

Speed unit support k and m.


You can also specify the maximum file size to download 


      
      

       
       $ curl URL --max-filesize 2500000
      
      

      
      
    
       
       
  • 1.
    •

The file size unit here is byte.


curl The command also supports authentication 


      
      

       
       $ curl -u username:password URL
      
      

      
      
    
       
       
  • 1.
    •

If you don't write password

      
      

       
       $ curl -u username URL
      
      

      
      
    
       
       
  • 1.
    •

Then knock back , You will be prompted to enter the password .


原网站

版权声明
本文为[51CTO]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/203/202207211532572550.html

边栏推荐

猜你喜欢

随机推荐