RESTful api and JSON POST data with curl

[Brent Zeiben]

Hi,
I am using a python script to organize some data and try to send it into
web2py via the @request.restful() api. In the python script the text is
mainly input from a free form text field. ( I believe some Unicode
characters are in there as well, however I remove everything that is ord
128 and above.)

My problem arises when the text contains an ampersand (&) and probably
anything that isn't allowed in without the form encoding. I thought python
json would escape the character some way but it does not seem to.

I am trying to use json to transfer this data to web2py via the curl
command within python.

Python Script

import json
import subprocess


url = 'http://127.0.0.1:8000/testing/default/api/people.json'
data = {"name":"My Full Name","biography":"Some simple information\n about
me & blah blah blah"}


jsondata = json.dumps(data)
result = subprocess.Popen(['curl',
'--user','username:password',
'-d',
jsondata,
url], stderr=subprocess.PIPE,stdout=subprocess.PIPE).communicate()[0]





The web2py relevant parts

db.py

db.define_table('people',
Field('name','string',length=200,requires=IS_NOT_EMPTY()),
Field('biography','text'))


default.py


auth.settings.allow_basic_login=True
@auth.requires_login()
@request.restful()
def api():
response.view = 'generic.' + request.extension
def POST(table_name,**vars):
if table_name == 'people':
return db.people.validate_and_insert(**vars)
else:
raise HTTP(400)
return locals()


So when the python script sends the data web2py complains because of the
escaping issue with the & in the data. I was doing some searching and
thought that changing the Content Type in the curl command to
application/json would help with this but then web2py has nothing in the
vars variable.

Thank you,
Brent

[Niphlod]

uhm... I may be wrong but.... request.restful accepts the args as a normal
page, that is a application/x-www-form-urlencoded or a multipart/form-data
. with curl would be something like curl -d name=blablabla -d
otherparameter=blablabla http://theurl

Post by Brent Zeiben
Hi,
I am using a python script to organize some data and try to send it into
mainly input from a free form text field. ( I believe some Unicode
characters are in there as well, however I remove everything that is ord
128 and above.)
My problem arises when the text contains an ampersand (&) and probably
anything that isn't allowed in without the form encoding. I thought python
json would escape the character some way but it does not seem to.
I am trying to use json to transfer this data to web2py via the curl
command within python.
Python Script
import json
import subprocess
url = 'http://127.0.0.1:8000/testing/default/api/people.json'
data = {"name":"My Full Name","biography":"Some simple information\n
about me & blah blah blah"}
jsondata = json.dumps(data)
result = subprocess.Popen(['curl',
'--user','username:password',
'-d',
jsondata,
url], stderr=subprocess.PIPE,stdout=subprocess.PIPE).communicate()[0]
The web2py relevant parts
db.py
db.define_table('people',
Field('name','string',length=200,requires=IS_NOT_EMPTY()),
Field('biography','text'))
default.py
auth.settings.allow_basic_login=True
@auth.requires_login()
@request.restful()
response.view = 'generic.' + request.extension
return db.people.validate_and_insert(**vars)
raise HTTP(400)
return locals()
So when the python script sends the data web2py complains because of the
escaping issue with the & in the data. I was doing some searching and
thought that changing the Content Type in the curl command to
application/json would help with this but then web2py has nothing in the
vars variable.
Thank you,
Brent

[Brent Zeiben]

Ok Thanks Niphlod

Thought I could use json to prepare the data, using urllib.urlencode on the
dictionary instead. Didn't have to change the curl command.

Thanks again.

Brent

Post by Niphlod
uhm... I may be wrong but.... request.restful accepts the args as a normal
page, that is a application/x-www-form-urlencoded or a multipart/form-data
. with curl would be something like curl -d name=blablabla -d
otherparameter=blablabla http://theurl

Post by Brent Zeiben
Hi,
I am using a python script to organize some data and try to send it into
mainly input from a free form text field. ( I believe some Unicode
characters are in there as well, however I remove everything that is ord
128 and above.)
My problem arises when the text contains an ampersand (&) and probably
anything that isn't allowed in without the form encoding. I thought python
json would escape the character some way but it does not seem to.
I am trying to use json to transfer this data to web2py via the curl
command within python.
Python Script
import json
import subprocess
url = 'http://127.0.0.1:8000/testing/default/api/people.json'
data = {"name":"My Full Name","biography":"Some simple information\n
about me & blah blah blah"}
jsondata = json.dumps(data)
result = subprocess.Popen(['curl',
'--user','username:password',
'-d',
jsondata,
url], stderr=subprocess.PIPE,stdout=subprocess.PIPE).communicate()[0]
The web2py relevant parts
db.py
db.define_table('people',
Field('name','string',length=200,requires=IS_NOT_EMPTY()),
Field('biography','text'))
default.py
auth.settings.allow_basic_login=True
@auth.requires_login()
@request.restful()
response.view = 'generic.' + request.extension
return db.people.validate_and_insert(**vars)
raise HTTP(400)
return locals()
So when the python script sends the data web2py complains because of the
escaping issue with the & in the data. I was doing some searching and
thought that changing the Content Type in the curl command to
application/json would help with this but then web2py has nothing in the
vars variable.
Thank you,
Brent

[Michele Comitini]

Brent,

you can read the raw body of a POST using the variable request.body

mic

Post by Brent Zeiben
Ok Thanks Niphlod
Thought I could use json to prepare the data, using urllib.urlencode on
the dictionary instead. Didn't have to change the curl command.
Thanks again.
Brent

Post by Niphlod
uhm... I may be wrong but.... request.restful accepts the args as a
normal page, that is a application/x-www-form-**urlencoded or a
multipart/form-data . with curl would be something like curl -d
name=blablabla -d otherparameter=blablabla http://theurl

Post by Brent Zeiben
Hi,
I am using a python script to organize some data and try to send it into
mainly input from a free form text field. ( I believe some Unicode
characters are in there as well, however I remove everything that is ord
128 and above.)
My problem arises when the text contains an ampersand (&) and probably
anything that isn't allowed in without the form encoding. I thought python
json would escape the character some way but it does not seem to.
I am trying to use json to transfer this data to web2py via the curl
command within python.
Python Script
import json
import subprocess
url = 'http://127.0.0.1:8000/**testing/default/api/people.**json<http://127.0.0.1:8000/testing/default/api/people.json>
'
data = {"name":"My Full Name","biography":"Some simple information\n
about me & blah blah blah"}
jsondata = json.dumps(data)
result = subprocess.Popen(['curl',
'--user','username:password',
'-d',
jsondata,
url], stderr=subprocess.PIPE,stdout=**subprocess.PIPE).communicate()*
*[0]
The web2py relevant parts
db.py
db.define_table('people',
Field('name','string',length=2**00,requires=IS_NOT_EMPTY()),
Field('biography','text'))
default.py
auth.settings.allow_basic_**login=True
@auth.requires_login()
@request.restful()
response.view = 'generic.' + request.extension
return db.people.validate_and_insert(****vars)
raise HTTP(400)
return locals()
So when the python script sends the data web2py complains because of the
escaping issue with the & in the data. I was doing some searching and
thought that changing the Content Type in the curl command to
application/json would help with this but then web2py has nothing in the
vars variable.
Thank you,
Brent

--

---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/groups/opt_out.