How to query a restful webservice using Python

How to query a restful webservice using Python

Writing a Python script that uses Requests lib to fire off a request to a
remote webservice. Here is my code (test.py):
import logging.config
from requests import Request, Session
logging.config.fileConfig('../../resources/logging.conf')
logr = logging.getLogger('pyLog')
url = 'https://158.74.36.11:7443/hqu/hqapi1/user/get.hqu'
token01 = 'hqstatus_python'
token02 = 'ytJFRyV7g'
response_length = 351
def main():
try:
logr.info('start SO example')
s = Session()
prepped = Request('GET', url, auth=(token01, token02),
params={'name': token01}).prepare()
response = s.send(prepped, stream=True, verify=False)
logr.info('status: ' + str(response.status_code))
logr.info('elapsed: ' + str(response.elapsed))
logr.info('headers: ' + str(response.headers))
logr.info('content: ' + response.raw.read(response_length).decode())
except Exception:
logr.exception("Exception")
finally:
logr.info('stop')
if __name__ == '__main__':
main()
I get the following successful output when i run this:
INFO test - start SO example
INFO test - status: 200
INFO test - elapsed: 0:00:00.532053
INFO test - headers: CaseInsensitiveDict({'server': 'Apache-Coyote/1.1',
'set-cookie': 'JSESSIONID=8F87A69FB2B92F3ADB7F8A73E587A10C; Path=/;
Secure; HttpOnly', 'content-type': 'text/xml;charset=UTF-8',
'transfer-encoding': 'chunked', 'date': 'Wed, 18 Sep 2013 06:34:28 GMT'})
INFO test - content: <?xml version="1.0" encoding="utf-8"?>
<UserResponse><Status>Success</Status> .... </UserResponse>
INFO test - stop
As you can see, there is this weird variable 'response_length' that i need
to pass to the response object (optional argument) to be able to read the
content. This variable has to be set to a numeric value that is equal to
length of the 'content'. This obviously means that i need to know the
response-content-length before hand, which is unreasonable.
If i don't pass that variable or set it to a value greater than the
content length, I get the following error:
Traceback (most recent call last):
File "\Python33\lib\http\client.py", line 590, in _readall_chunked
chunk_left = self._read_next_chunk_size()
File "\Python33\lib\http\client.py", line 562, in _read_next_chunk_size
return int(line, 16)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb4 in position 0:
invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test.py", line 22, in main
logr.info('content: ' + response.raw.read().decode())
File
"\Python33\lib\site-packages\requests\packages\urllib3\response.py",
line 167, in read
data = self._fp.read()
File "\Python33\lib\http\client.py", line 509, in read
return self._readall_chunked()
File "\Python33\lib\http\client.py", line 594, in _readall_chunked
raise IncompleteRead(b''.join(value))
http.client.IncompleteRead: IncompleteRead(351 bytes read)
How do i make this work without this 'response_length' variable? Also, are
there any better options than 'Requests' lib?
PS: this code is an independent script, and does not run in the Django
framework.