Dbtool Does Nothing. Still Have Odbc Data Width Errors

I ran dbtool and it appears to do nothing. As soon as I give an answer to the prompt about verbose level, I IMMEDIATELY get back to the linux prompt as if the tool has run and done everything in a nanosecond.

It is actually doing nothing as I the ODBC error messages continue.

I also noted that it didn't care whether I chose 0 for single user or 1 for self-service or 2 (for more threads). It seems like this tool does nothing. (to be clear, I ran dbtool with the dbname as the parameter on the command line, then I chose option for SQL Width number 2 (I also chose option 1, but no difference).
 
Last edited:

TomBascom

Curmudgeon
Dbtool does in fact work, so you have some other issue. What it your version?
OpenEdge version is 10.2B07 on Red Hat Enterprise Linux Server release 6.5

I did not realize that dbtool does not display any error messages on screen when it fails.
So I found that it created a file called mydatabasename.lg
When I looked there it gave me the idea that maybe I haven't specified my database correctly (I didn't include a pathname because an example of running dbtool on progress.com implied that is what you do). But QAD has a page where it indicates the path is needed
(QAD Business Intelligence Documentation)
So I then ran
dbtool /mydirectorytreeofpaths/mydatabasename (with no .db extension)
nothing displayed onscreen, but now I know I need to look in my current directory for files to learn what happened. This generated a file called protrace.16274 which running cat on it displayed this on screen:

#1 [0x4584d8] dbut_uttraceback+0x144 from /apps/dlc/bin/dbtool
#2 [0x455d78] dbut_uttrace+0x13d from /apps/dlc/bin/dbtool
#3 [0x4441bd] dbExit+0x4d from /apps/dlc/bin/dbtool
#4 [0x45d6d7] dsmFatalMsgnCallBack+0x10b from /apps/dlc/bin/dbtool
#5 [0x4a6918] shmmsg+0x58 from /apps/dlc/bin/dbtool
#6 [0x4a60b3] shmAttachSegment+0x1ed from /apps/dlc/bin/dbtool
#7 [0x4a6122] shmVerifySegment+0x4e from /apps/dlc/bin/dbtool
#8 [0x4a6389] shmattsegs+0x118 from /apps/dlc/bin/dbtool
#9 [0x4a5701] shmAttach+0x31 from /apps/dlc/bin/dbtool
#10 [0x449eda] dbenv2+0x12d from /apps/dlc/bin/dbtool
#11 [0x4488e9] dbenv+0x69 from /apps/dlc/bin/dbtool
#12 [0x466f1b] dsmUserSecureConnect+0x7da from /apps/dlc/bin/dbtool
#13 [0x411b0f] myConnect+0x3f4 from /apps/dlc/bin/dbtool
#14 [0x40b71b] main+0x389 from /apps/dlc/bin/dbtool
#15 [0x77e1ed1d] __tls_get_addr@@GLIBC_2.3+0x77e1ed1d from /lib64/libc.so.6
 
Last edited:

TomBascom

Curmudgeon
The auto truncate options prevent ODBC from reporting errors if the data is too wide for ODBC's idea of what it should be.

Whether that is the "correct" choice or not is in the eye of the beholder. Over the years many people have said that's what they wanted though.
 
The auto truncate options prevent ODBC from reporting errors if the data is too wide for ODBC's idea of what it should be.
ODBC generated an error on my environment when I tried to connect after I added this option to the DSN

[DataDirect][ODBC Progress OpenEdge Wire Protocol driver][OPENEDGE]Invalid Connection Property in Connection Property string

The ODBC driver is the one you can currently download from QAD which is for 11.6 and earlier Progress databases (though it wouldn't surprise me if this specific scenario of special settings and versions of drivers and DBs has not been verified to work). Specifically the ODBC Administrator lists this driver name as Progress OpenEdge 11.6 Driver and in the version column for the driver itself it lists 7.10.05.167.
 

Rob Fitzpatrick

ProgressTalk.com Sponsor
The options for truncating data and altering the meta-schema are only available starting in OE 11.5.

I'd suggest running dbtool manually first before resorting to some "automagic" option. I can't say for sure but based on the call stack, my semi-educated guess is that your userid doesn't have permission to attach to the database's shared memory segment(s). Can you connect a shared-memory client without errors? E.g. mpro /mydirectorytreeofpaths/mydatabasename
 
My employer is still only in the evaluation phase of QAD 2016 which uses Progress 11.6. For my need today the I might be able to use that evaluation database. For longer term knowing the 11.5 requirement might give some workaround. I'll explore that later. But for right now, it seems I need to debug the stack dump.

If you know QAD, then maybe saying that I am connecting using the same Linux user account that runs the QAD system (which I guess is the database creator/owner of the progress database I'm trying to run dbtool on.)

I guess that is why I was able to run your suggested command (mpro /mydirectorytreeofpaths/mydatabase and that environment came up. I don't know how to use that environment, so I don't know if I needed to do something more to check my system.

I did run sqlexp -db mydatabasename -S mydblistenerport
and then I could do something simple like Select top 10 * from sysprogress.systables
which if I had tried to do this via ODBC I would get some error about data widths being wrong.
 

Rob Fitzpatrick

ProgressTalk.com Sponsor
I managed to get this call stack, pretty close to yours, by creating a db as root, chowning the db log file so I had write permission on it, and running dbtool with my non-root userid:

Code:
#1   [0x455562] dbut_uttraceback+0x12b  from /u/DLC/11.6/oe/bin/dbtool  
#2   [0x45247b] dbut_uttrace+0x139  from /u/DLC/11.6/oe/bin/dbtool      
#3   [0x43d6f0] dbExit+0x47  from /u/DLC/11.6/oe/bin/dbtool              
#4   [0x459b22] dsmFatalMsgnCallBack+0x110  from /u/DLC/11.6/oe/bin/dbtool
#5   [0x4bc6e0] shmmsg+0x60  from /u/DLC/11.6/oe/bin/dbtool              
#6   [0x4bc1ac] shmAttachSegment+0x1eb  from /u/DLC/11.6/oe/bin/dbtool  
#7   [0x4bc229] shmVerifySegment+0x55  from /u/DLC/11.6/oe/bin/dbtool    
#8   [0x4bc3e1] shmattsegs+0x10c  from /u/DLC/11.6/oe/bin/dbtool        
#9   [0x4bbcde] shmAttach+0x31  from /u/DLC/11.6/oe/bin/dbtool          
#10  [0x445e7a] dbenv2+0x12c  from /u/DLC/11.6/oe/bin/dbtool            
#11  [0x447b98] dbenv+0x6a  from /u/DLC/11.6/oe/bin/dbtool              
#12  [0x46a729] dsmUserSecureConnect+0x7de  from /u/DLC/11.6/oe/bin/dbtool
#13  [0x417c18] myConnect+0x6f6  from /u/DLC/11.6/oe/bin/dbtool          
#14  [0x41491b] main+0x253  from /u/DLC/11.6/oe/bin/dbtool              
#15  [0xb3a1ed5d] __rpc_thread_destroy+0xffedd8ad  from /lib64/libc.so.6
 

Rob Fitzpatrick

ProgressTalk.com Sponsor
Odd. If you are using the account that created (and presumably owns) the .db file then those same permissions would be on the shared memory segments. You can see those permissions with ipcs -m (cross-reference with proutil -C dbipcs output to get the right shmid). With that user you should be able to run dbtool successfully.
 

Rob Fitzpatrick

ProgressTalk.com Sponsor
I don't see any KB articles that seem related to this issue.

Not sure what else to suggest, apart from trying to run dbtool from the application if that's an option, or else contact QAD for support.
 
Odd. If you are using the account that created (and presumably owns) the .db file then those same permissions would be on the shared memory segments. You can see those permissions with ipcs -m (cross-reference with proutil -C dbipcs output to get the right shmid). With that user you should be able to run dbtool successfully.
running ipcs from the account I've always used shows the owner for the shared memory segment of the process running the db I am trying to check (using that cross reference suggestion you made to locate the shmid) is root with perms=644 and nattch=14 (I don't know what perms and nattch are).
this account is in the /etc/sudoers file in the root section of the file and running
sudo -s and getting the shell, then issuing the the id command. I see uid=0(root) gid=0(root) groups=0(root).

So this account has root access. Since I wasn't around to create the database, maybe there is a way to create it with some other user account which somehow causes a problem??? I am grasping at straws here though.

As for QAD, as I looked farther, it just lists doing manual dbtool commands at the command line, so while it has slightly more documentation then I found on progress, it still doesn't work or begin to cover the problems I am seeing here with this stack dump.
 
OK. I seemed to have solved the problem by logging on with the root user. I'm not sure why the account which seemed to have root privileges was not good enough, but I'll take this for now. I also just learned we are going live in about 2 months with QAD 2016 which uses Progress 11.6. So it won't be that long before I can just use that ODBC property to get around this issue.
 
So after running dbtools logged in as root I was able to use the ODBC driver in order to import some progress tables into SQL Server using SQL Server's default import facilities. However, one bug that I referenced here
Querying The Catalog Via Odbc
still remains. When I used linked servers to query the catalog sysprogress I still get errors that have to do with false reporting of data sizes. So I guess dbtool doesn't correct that. I will try the extra ODBC property with the non-production 11.6 database to see if that bug is fixed by that.

I did look briefly at Data Server for SQL Server but this looks to be a product for letting 4GL programs access SQL Server. It doesn't seem to be a bridge to push data more directly (e.g. data sync or at least sending progress data to a data warehouse in SQL Server) between the two DBMS'. Correct?
 

Rob Fitzpatrick

ProgressTalk.com Sponsor
root with perms=644 and nattch=14 (I don't know what perms and nattch are)
"Perms" is permissions: 6 (read/write) for owner, 4 (read) for group, and 4 (read) for the rest of the world. So in this case if you aren't root, you get read-only access to the shared-memory segment. This would explain not being able to run dbtool. I believe "nattch" is the number of processes attached to (i.e. that have mapped) that segment.

I did look briefly at Data Server for SQL Server but this looks to be a product for letting 4GL programs access SQL Server. It doesn't seem to be a bridge to push data more directly (e.g. data sync or at least sending progress data to a data warehouse in SQL Server) between the two DBMS'. Correct?
Yes, DataServers are for running an OpenEdge ABL application with a non-OpenEdge database.
 
I thought I saw that QAD gets started by the account I logged in with. But apparently it gets launched by the root, which to me is suboptimal. Later down the road I will do a security audit on our QAD installation and likely change a bunch of stuff.
 

cj_brandt

Active Member
If you are new to OE / Progress databases and are more familiar with MS SQL, you might consider exporting the data from OE and loading it into MS SQL to do your data investigation.
 
Top