Python requests上传文件实现步骤

2023-07-16 17:18:03 335

官方文档：https://2.python-requests.org//en/master/

工作中涉及到一个功能，需要上传附件到一个接口，接口参数如下：

使用httppost提交附件multipart/form-data格式，url:http://test.com/flow/upload，

字段列表：
md5://md5加密（随机值_当时时间戳）
filesize://文件大小
file://文件内容(须含文件名)
返回值：
{"success":true,"uploadName":"tmp.xml","uploadPath":"uploads\/201311\/758e875fb7c7a508feef6b5036119b9f"}

由于工作中主要用python，并且项目中已有使用requests库的地方，所以计划使用requests来实现，本来以为是很简单的一个小功能，结果花费了大量的时间，requests官方的例子只提到了上传文件，并不需要传额外的参数：

https://2.python-requests.org//en/master/user/quickstart/#post-a-multipart-encoded-file

>>>url='https://httpbin.org/post'
>>>files={'file':('report.xls',open('report.xls','rb'),'application/vnd.ms-excel',{'Expires':'0'})}

>>>r=requests.post(url,files=files)
>>>r.text
{
...
"files":{
"file":""
},
...
}

但是如果涉及到了参数的传递时，其实就要用到requests的两个参数：data、files，将要上传的文件传入files，将其他参数传入data，request库会将两者合并到一起做一个multipart，然后发送给服务器。

最终实现的代码是这样的：

withopen(file_name)asf:
content=f.read()
request_data={
'md5':md5.md5('%d_%d'%(0,int(time.time()))).hexdigest(),
'filesize':len(content),
}
files={'file':(file_name,open(file_name,'rb'))}
MyLogger().getlogger().info('url:%s'%(request_url))
resp=requests.post(request_url,data=request_data,files=files)

虽然最终代码可能看起来很简单，但是其实我费了好大功夫才确认这样是OK的，中间还翻了requests的源码，下面记录一下翻阅源码的过程：

首先，找到post方法的实现，在requests.api.py中：

defpost(url,data=None,json=None,**kwargs):
r"""SendsaPOSTrequest.

:paramurl:URLforthenew:class:`Request`object.
:paramdata:(optional)Dictionary,listoftuples,bytes,orfile-like
objecttosendinthebodyofthe:class:`Request`.
:paramjson:(optional)jsondatatosendinthebodyofthe:class:`Request`.
:param\*\*kwargs:Optionalargumentsthat``request``takes.
:return::class:`Response`object
:rtype:requests.Response
"""

returnrequest('post',url,data=data,json=json,**kwargs)

这里可以看到它调用了request方法，咱们继续跟进request方法，在requests.api.py中：

defrequest(method,url,**kwargs):
"""Constructsandsendsa:class:`Request`.

:parammethod:methodforthenew:class:`Request`object:``GET``,``OPTIONS``,``HEAD``,``POST``,``PUT``,``PATCH``,or``DELETE``.
:paramurl:URLforthenew:class:`Request`object.
:paramparams:(optional)Dictionary,listoftuplesorbytestosend
inthequerystringforthe:class:`Request`.
:paramdata:(optional)Dictionary,listoftuples,bytes,orfile-like
objecttosendinthebodyofthe:class:`Request`.
:paramjson:(optional)AJSONserializablePythonobjecttosendinthebodyofthe:class:`Request`.
:paramheaders:(optional)DictionaryofHTTPHeaderstosendwiththe:class:`Request`.
:paramcookies:(optional)DictorCookieJarobjecttosendwiththe:class:`Request`.
:paramfiles:(optional)Dictionaryof``'name':file-like-objects``(or``{'name':file-tuple}``)formultipartencodingupload.
``file-tuple``canbea2-tuple``('filename',fileobj)``,3-tuple``('filename',fileobj,'content_type')``
ora4-tuple``('filename',fileobj,'content_type',custom_headers)``,where``'content-type'``isastring
definingthecontenttypeofthegivenfileand``custom_headers``adict-likeobjectcontainingadditionalheaders
toaddforthefile.
:paramauth:(optional)AuthtupletoenableBasic/Digest/CustomHTTPAuth.
:paramtimeout:(optional)Howmanysecondstowaitfortheservertosenddata
beforegivingup,asafloat,ora:ref:`(connecttimeout,read
timeout)`tuple.
:typetimeout:floatortuple
:paramallow_redirects:(optional)Boolean.Enable/disableGET/OPTIONS/POST/PUT/PATCH/DELETE/HEADredirection.Defaultsto``True``.
:typeallow_redirects:bool
:paramproxies:(optional)DictionarymappingprotocoltotheURLoftheproxy.
:paramverify:(optional)Eitheraboolean,inwhichcaseitcontrolswhetherweverify
theserver'sTLScertificate,orastring,inwhichcaseitmustbeapath
toaCAbundletouse.Defaultsto``True``.
:paramstream:(optional)if``False``,theresponsecontentwillbeimmediatelydownloaded.
:paramcert:(optional)ifString,pathtosslclientcertfile(.pem).IfTuple,('cert','key')pair.
:return::class:`Response`object
:rtype:requests.Response

Usage::

>>>importrequests
>>>req=requests.request('GET','https://httpbin.org/get')

"""

#Byusingthe'with'statementwearesurethesessionisclosed,thuswe
#avoidleavingsocketsopenwhichcantriggeraResourceWarninginsome
#cases,andlooklikeamemoryleakinothers.
withsessions.Session()assession:
returnsession.request(method=method,url=url,**kwargs)

这个方法的注释比较多，从注释里其实已经可以看到files参数使用传送文件，但是还是无法知道当需要同时传递参数和文件时该如何处理，继续跟进session.request方法，在requests.session.py中：

defrequest(self,method,url,
params=None,data=None,headers=None,cookies=None,files=None,
auth=None,timeout=None,allow_redirects=True,proxies=None,
hooks=None,stream=None,verify=None,cert=None,json=None):
"""Constructsa:class:`Request`,preparesitandsendsit.
Returns:class:`Response`object.

:parammethod:methodforthenew:class:`Request`object.
:paramurl:URLforthenew:class:`Request`object.
:paramparams:(optional)Dictionaryorbytestobesentinthequery
stringforthe:class:`Request`.
:paramdata:(optional)Dictionary,listoftuples,bytes,orfile-like
objecttosendinthebodyofthe:class:`Request`.
:paramjson:(optional)jsontosendinthebodyofthe
:class:`Request`.
:paramheaders:(optional)DictionaryofHTTPHeaderstosendwiththe
:class:`Request`.
:paramcookies:(optional)DictorCookieJarobjecttosendwiththe
:class:`Request`.
:paramfiles:(optional)Dictionaryof``'filename':file-like-objects``
formultipartencodingupload.
:paramauth:(optional)Authtupleorcallabletoenable
Basic/Digest/CustomHTTPAuth.
:paramtimeout:(optional)Howlongtowaitfortheservertosend
databeforegivingup,asafloat,ora:ref:`(connecttimeout,
readtimeout)`tuple.
:typetimeout:floatortuple
:paramallow_redirects:(optional)SettoTruebydefault.
:typeallow_redirects:bool
:paramproxies:(optional)Dictionarymappingprotocolorprotocoland
hostnametotheURLoftheproxy.
:paramstream:(optional)whethertoimmediatelydownloadtheresponse
content.Defaultsto``False``.
:paramverify:(optional)Eitheraboolean,inwhichcaseitcontrolswhetherweverify
theserver'sTLScertificate,orastring,inwhichcaseitmustbeapath
toaCAbundletouse.Defaultsto``True``.
:paramcert:(optional)ifString,pathtosslclientcertfile(.pem).
IfTuple,('cert','key')pair.
:rtype:requests.Response
"""
#CreatetheRequest.
req=Request(
method=method.upper(),
url=url,
headers=headers,
files=files,
data=dataor{},
json=json,
params=paramsor{},
auth=auth,
cookies=cookies,
hooks=hooks,
)
prep=self.prepare_request(req)

proxies=proxiesor{}

settings=self.merge_environment_settings(
prep.url,proxies,stream,verify,cert
)

#Sendtherequest.
send_kwargs={
'timeout':timeout,
'allow_redirects':allow_redirects,
}
send_kwargs.update(settings)
resp=self.send(prep,**send_kwargs)

returnresp

先大概看一下这个方法，先是准备request，最后一步是调用send，推测应该是发送请求了，所以我们需要跟进到prepare_request方法中，在requests.session.py中：

defprepare_request(self,request):
"""Constructsa:class:`PreparedRequest`for
transmissionandreturnsit.The:class:`PreparedRequest`hassettings
mergedfromthe:class:`Request`instanceandthoseofthe
:class:`Session`.

:paramrequest::class:`Request`instancetopreparewiththis
session'ssettings.
:rtype:requests.PreparedRequest
"""
cookies=request.cookiesor{}

#BootstrapCookieJar.
ifnotisinstance(cookies,cookielib.CookieJar):
cookies=cookiejar_from_dict(cookies)

#Mergewithsessioncookies
merged_cookies=merge_cookies(
merge_cookies(RequestsCookieJar(),self.cookies),cookies)

#Setenvironment'sbasicauthenticationifnotexplicitlyset.
auth=request.auth
ifself.trust_envandnotauthandnotself.auth:
auth=get_netrc_auth(request.url)

p=PreparedRequest()
p.prepare(
method=request.method.upper(),
url=request.url,
files=request.files,
data=request.data,
json=request.json,
headers=merge_setting(request.headers,self.headers,dict_class=CaseInsensitiveDict),
params=merge_setting(request.params,self.params),
auth=merge_setting(auth,self.auth),
cookies=merged_cookies,
hooks=merge_hooks(request.hooks,self.hooks),
)
returnp

在prepare_request中，生成了一个PreparedRequest对象，并调用其prepare方法，跟进到prepare方法中，在requests.models.py中：

defprepare(self,
method=None,url=None,headers=None,files=None,data=None,
params=None,auth=None,cookies=None,hooks=None,json=None):
"""Preparestheentirerequestwiththegivenparameters."""

self.prepare_method(method)
self.prepare_url(url,params)
self.prepare_headers(headers)
self.prepare_cookies(cookies)
self.prepare_body(data,files,json)
self.prepare_auth(auth,url)

#Notethatprepare_authmustbelasttoenableauthenticationschemes
#suchasOAuthtoworkonafullypreparedrequest.

#ThisMUSTgoafterprepare_auth.Authenticatorscouldaddahook
self.prepare_hooks(hooks)

这里调用许多prepare_xx方法，这里我们只关心处理了data、files、json的方法，跟进到prepare_body中，在requests.models.py中：

defprepare_body(self,data,files,json=None):
"""PreparesthegivenHTTPbodydata."""

#Checkiffile,fo,generator,iterator.
#Ifnot,runthroughnormalprocess.

#Nottin'onyou.
body=None
content_type=None

ifnotdataandjsonisnotNone:
#urllib3requiresabytes-likebody.Python2'sjson.dumps
#providesthisnatively,butPython3givesaUnicodestring.
content_type='application/json'
body=complexjson.dumps(json)
ifnotisinstance(body,bytes):
body=body.encode('utf-8')

is_stream=all([
hasattr(data,'__iter__'),
notisinstance(data,(basestring,list,tuple,Mapping))
])

try:
length=super_len(data)
except(TypeError,AttributeError,UnsupportedOperation):
length=None

ifis_stream:
body=data

ifgetattr(body,'tell',None)isnotNone:
#Recordthecurrentfilepositionbeforereading.
#Thiswillallowustorewindafileintheevent
#ofaredirect.
try:
self._body_position=body.tell()
except(IOError,OSError):
#ThisdifferentiatesfromNone,allowingustocatch
#afailed`tell()`laterwhentryingtorewindthebody
self._body_position=object()

iffiles:
raiseNotImplementedError('Streamedbodiesandfilesaremutuallyexclusive.')

iflength:
self.headers['Content-Length']=builtin_str(length)
else:
self.headers['Transfer-Encoding']='chunked'
else:
#Multi-partfileuploads.
iffiles:
(body,content_type)=self._encode_files(files,data)
else:
ifdata:
body=self._encode_params(data)
ifisinstance(data,basestring)orhasattr(data,'read'):
content_type=None
else:
content_type='application/x-www-form-urlencoded'

self.prepare_content_length(body)

#Addcontent-typeifitwasn'texplicitlyprovided.
ifcontent_typeand('content-type'notinself.headers):
self.headers['Content-Type']=content_type

self.body=body

这个函数比较长，需要重点关注L52，这里调用了_encode_files方法，我们跟进这个方法：

def_encode_files(files,data):
"""Buildthebodyforamultipart/form-datarequest.

Willsuccessfullyencodefileswhenpassedasadictoralistof
tuples.Orderisretainedifdataisalistoftuplesbutarbitrary
ifparametersaresuppliedasadict.
Thetuplesmaybe2-tuples(filename,fileobj),3-tuples(filename,fileobj,contentype)
or4-tuples(filename,fileobj,contentype,custom_headers).
"""
if(notfiles):
raiseValueError("Filesmustbeprovided.")
elifisinstance(data,basestring):
raiseValueError("Datamustnotbeastring.")

new_fields=[]
fields=to_key_val_list(dataor{})
files=to_key_val_list(filesor{})

forfield,valinfields:
ifisinstance(val,basestring)ornothasattr(val,'__iter__'):
val=[val]
forvinval:
ifvisnotNone:
#Don'tcallstr()onbytestrings:inPy3itallgoeswrong.
ifnotisinstance(v,bytes):
v=str(v)

new_fields.append(
(field.decode('utf-8')ifisinstance(field,bytes)elsefield,
v.encode('utf-8')ifisinstance(v,str)elsev))

for(k,v)infiles:
#supportforexplicitfilename
ft=None
fh=None
ifisinstance(v,(tuple,list)):
iflen(v)==2:
fn,fp=v
eliflen(v)==3:
fn,fp,ft=v
else:
fn,fp,ft,fh=v
else:
fn=guess_filename(v)ork
fp=v

ifisinstance(fp,(str,bytes,bytearray)):
fdata=fp
elifhasattr(fp,'read'):
fdata=fp.read()
eliffpisNone:
continue
else:
fdata=fp

rf=RequestField(name=k,data=fdata,filename=fn,headers=fh)
rf.make_multipart(content_type=ft)
new_fields.append(rf)

body,content_type=encode_multipart_formdata(new_fields)

returnbody,content_type

OK，到此为止，仔细阅读完这个段代码，就可以搞明白requests.post方法传入的data、files两个参数的作用了，其实requests在这里把它俩合并在一起了，作为post的body。

以上就是本文的全部内容，希望对大家的学习有所帮助，也希望大家多多支持毛票票。

声明：本文内容来源于网络，版权归原作者所有，内容由互联网用户自发贡献自行上传，本网站不拥有所有权，未作人工编辑处理，也不承担相关法律责任。如果您发现有涉嫌版权的内容，欢迎发送邮件至：czq8825#qq.com（发邮件时，请将#更换为@）进行举报，并提供相关证据，一经查实，本站将立刻删除涉嫌侵权内容。

Python requests上传文件实现步骤

热门推荐

随机推荐