Error in trunk opening PostgreSQL database

[ron_m]

I tested my app with version 1.92.1 and it works. I then copied it to trunk
and I can no longer connect to the PostgreSQL database I use.

I looked at the code in dal.py for the PostgreSQL adapter and driver_args
defaults to empty dict {}, adding some debug print code shows it is set on
the class construction to be {'check_same_thread': False} and when this is
passed to the driver an invalid keyword argument error occurs

Here is a traceback from the ticket.

web2py™ Version 1.92.1 (2011-03-03 08:26:32) Python Python 2.6.5:
/usr/bin/python Traceback

1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

Traceback (most recent call last):
File "/home/camcentral/Dev/web2py-hg/gluon/restricted.py", line 188, in restricted
exec ccode in environment
File "/home/camcentral/Dev/web2py-hg/applications/ccims/models/A_db.py" <http://127.0.0.1:8000/admin/default/edit/ccims/models/A_db.py>, line 22, in <module>
db = DAL('postgres://user:***@localhost/cc_ims', POOL_SIZE, check_reserved=check_reserved_value)
File "/home/camcentral/Dev/web2py-hg/gluon/dal.py", line 3529, in __init__
raise RuntimeError, "Failure to connect, tried 5 times:\n%s" % error
RuntimeError: Failure to connect, tried 5 times:
'check_same_thread' is an invalid keyword argument for this function

Error snapshot [image: help] Detailed traceback description

<type 'exceptions.RuntimeError'>(Failure to connect, tried 5 times:
'check_same_thread' is an invalid keyword argument for this function)
Frames

-

*File /home/camcentral/Dev/web2py-hg/gluon/restricted.py in restricted at
line 188* code arguments variables
Function argument list

(code='# -*- coding: utf-8 -*- \n# this file is
released...sion.table_name.requires = IS_IN_SET(db.tables)\n\n',
environment={'A': <class 'gluon.html.A'>, 'B': <class 'gluon.html.B'>,
'BEAUTIFY': <class 'gluon.html.BEAUTIFY'>, 'BODY': <class
'gluon.html.BODY'>, 'BR': <class 'gluon.html.BR'>, 'CENTER': <class
'gluon.html.CENTER'>, 'CLEANUP': <class 'gluon.validators.CLEANUP'>, 'CODE':
<class 'gluon.html.CODE'>, 'COUPLE_MENUS': True, 'CRYPT': <class
'gluon.validators.CRYPT'>, ...},
layer='/home/camcentral/Dev/web2py-hg/applications/ccims/models/A_db.py')
Code listing

183.
184.
185.
186.
187.
188.

189.
190.
191.
192.

if type(code) == types.CodeType:
ccode = code
else:
ccode = compile2(code,layer)

exec ccode in environment

except HTTP:
raise
except Exception:
# XXX Show exception in Wing IDE if running in debugger

Variables environment {'A': <class 'gluon.html.A'>, 'B': <class
'gluon.html.B'>, 'BEAUTIFY': <class 'gluon.html.BEAUTIFY'>, 'BODY': <class
'gluon.html.BODY'>, 'BR': <class 'gluon.html.BR'>, 'CENTER': <class
'gluon.html.CENTER'>, 'CLEANUP': <class 'gluon.validators.CLEANUP'>, 'CODE':
<class 'gluon.html.CODE'>, 'COUPLE_MENUS': True, 'CRYPT': <class
'gluon.validators.CRYPT'>, ...} ccode <code object <module> at
0x30bda08, file "/home/...py-hg/applications/ccims/models/A_db.py", line 8>
-

*File /home/camcentral/Dev/web2py-hg/applications/ccims/models/A_db.py in
<module> at line 22* code arguments variables
Function argument list

()
Code listing

17.
18.
19.
20.
21.
22.

23.
24.
25.
26.

# else use a normal relational database
#*** Add customization
# db = DAL('sqlite://storage.sqlite') # if not, use SQLite or other DB
# CamCentral IMS configuration database
# db = DAL('mysql://root:***@localhost/cc_ims', POOL_SIZE, check_reserved=check_reserved_value)
db = DAL('postgres://user:***@localhost/cc_ims', POOL_SIZE, check_reserved=check_reserved_value)

## if no need for session
# session.forget()

#########################################################################

Variables DAL <class 'gluon.dal.DAL'> check_reserved undefined
POOL_SIZE 0 db undefined check_reserved_value ['all']
-

*File /home/camcentral/Dev/web2py-hg/gluon/dal.py in __init__ at line
3529* code arguments variables
Function argument list

(self=<DAL {'_lastsql': '', '_db_codec': 'UTF-8', '_ur...cc_ims',
'_pool_size': 0, '_dbname': 'postgres'}>,
uri='postgres://user:***@localhost/cc_ims', pool_size=0, folder=None,
db_codec='UTF-8', check_reserved=['all'], migrate=True, fake_migrate=False,
decode_credentials=False, driver_args={'check_same_thread': False})

[ron_m]

Additional information:

I added a print to the front end of DAL.__init__ and was surprised to find
driver_args was set to {'check_same_thread': False} but I did not set it in
my db = DAL() class constructor call.

So here is the issue. I start up the development environment on Linux with a
terminal command "python web2py.py" sitting in the correct web2py directory.
Because I have Tcl/TK installed on the system it comes up, solicits an admin
password and then I start the server.

This in turn brings up a browser window running the welcome application. The
welcome application as written uses SQLLite as the database and since it is
default installed in Python >= 2.5 I have the driver and a connection
succeeds. The SQLLite adapter does use this driver_args parameter and if it
is not set forces it to the value I am seeing.

Somehow driver_args gets stored globally and becomes the default for all
other adapter calls to __init__ on any other driver as well.

To test this theory I commented out the db=DAL(...) line in db.py in the
welcome application and restarted the server. This caused the welcome app to
crash during the TclTk solicited start of the welcome app when the browser
is opened but at least the DAL did not get initialized using the SQLLite
database.

Now I am able to run my own application and the driver_args = {} which would
be correct for the PostgreSQL adapter.

Hope that helps isolate it a little better.

Ron

[Massimo Di Pierro]

You must be using trunk. I think this is a bug in trunk but not
stable. Thanks for reporting it.

Massimo

Post by ron_m
I added a print to the front end of DAL.__init__ and was surprised to find
driver_args was set to {'check_same_thread': False} but I did not set it in
my db = DAL() class constructor call.
So here is the issue. I start up the development environment on Linux with a
terminal command "python web2py.py" sitting in the correct web2py directory.
Because I have Tcl/TK installed on the system it comes up, solicits an admin
password and then I start the server.
This in turn brings up a browser window running the welcome application. The
welcome application as written uses SQLLite as the database and since it is
default installed in Python >= 2.5 I have the driver and a connection
succeeds. The SQLLite adapter does use this driver_args parameter and if it
is not set forces it to the value I am seeing.
Somehow driver_args gets stored globally and becomes the default for all
other adapter calls to __init__ on any other driver as well.
To test this theory I commented out the db=DAL(...) line in db.py in the
welcome application and restarted the server. This caused the welcome app to
crash during the TclTk solicited start of the welcome app when the browser
is opened but at least the DAL did not get initialized using the SQLLite
database.
Now I am able to run my own application and the driver_args = {} which would
be correct for the PostgreSQL adapter.
Hope that helps isolate it a little better.
Ron

[ron_m]

I just downloaded the nightly build, unpacked it, copied over my application
and get the same problem.
I don't have a problem with 1.92.1. I don't normally use trunk but test it
once in a while to see what is coming. :-)

Ron

[Massimo Di Pierro]

please try change line in dal

args =
(self,uri,pool_size,folder,db_codec,credential_decoder,driver_args)

to

args =
(self,uri,pool_size,folder,db_codec,credential_decoder,driver_args or
{})

Post by ron_m
I just downloaded the nightly build, unpacked it, copied over my application
and get the same problem.
I don't have a problem with 1.92.1. I don't normally use trunk but test it
once in a while to see what is coming. :-)
Ron

[ron_m]

Wow that fixed it. You are a Python Ninja :-)

I would sure like to know what the mechanism was that caused driver_args to
bleed from SQLLite to the other adapters even though it is defaulted it to
an empty dict on the class initializer. I also printed driver_args as the
first line of code in DAL.__init__() and it had the value left over from
SQLLite so how driver_args or {} produces {} when there is a value in
driver_args.

But you are very busy so if you don't have time no problem.

[Massimo Di Pierro]

class DAL:
def __init__(self,...,driver_args={})

so because this is defined in a module, every time driver_args is not
passed it is set to {}, the same {}. If one app changes, the default
changes for all apps. It is kind of counter intuitive but it works
that way.

It is very dangerous to set a default in a function to a list of
dictionary, because if somewhere downstream the object changed for one
instance, it changes the default for all future instances.

Massimo

Post by ron_m
Wow that fixed it. You are a Python Ninja :-)
I would sure like to know what the mechanism was that caused driver_args to
bleed from SQLLite to the other adapters even though it is defaulted it to
an empty dict on the class initializer. I also printed driver_args as the
first line of code in DAL.__init__() and it had the value left over from
SQLLite so how driver_args or {} produces {} when there is a value in
driver_args.
But you are very busy so if you don't have time no problem.

[ron_m]

Ah Ok, I was trying to reproduce with a simple program containing a couple
of classes with a main all in one file but it did not exhibit the problem
behaviour. So having the classes in a module is part of the recipe and lists
and dicts are passed around as object references to the container leaving
the contents open to modification.

[Massimo Di Pierro]

Post by ron_m

def f(a={}): return a

...

Post by ron_m

x=f()
x[1]=1
print x

{1: 1}

Post by ron_m

y=f()
print y

{1: 1}

Post by ron_m
Ah Ok, I was trying to reproduce with a simple program containing a couple
of classes with a main all in one file but it did not exhibit the problem
behaviour. So having the classes in a module is part of the recipe and lists
and dicts are passed around as object references to the container leaving
the contents open to modification.

[ron_m]

Thanks, it is easy to see how the default parameter gets changed with that
example since the binding passes back outside the function through the
return value, gets assigned to another variable and then the container
contents modified through that assignment.