Answered Schema Change on a UFT-8/Unicode Database

[Cecil]

In the past I've applied schema changes from Windows (WebSpeed Workshop for Windows) and the UTF-8 database is located on Linux. Historically you can't access a UTF-8 database in shared memory mode on Linux because of the fear of corruption, unless it is accessed in batch mode only.

Now I have my application and database 'in the cloud'. How do I apply my schema changes to my database if I can't use the CHUI Data Administrator tool? Is there a batch tool to apply schema delta file?

INFO: OE 11.3.2
OS: CentOS 6.5 64bit

 

[Stefan]

Connect with -S instead of through shared memory?

 

[RealHeavyDude]

I've never heard that you cannot connect to a database that uses UTF-8 code page via shared memory. As far as I know the ChUI client ( which does not include the AppServer or WebSpeed Broker ) does not support UTF-8. But you should still be able to connect with a ChUI client that uses iso8859-1 or the like to such a database via shared memorey. This should work for you unless the names you use in the schema object do contain characters that are not present in that code page.

I've done it over and over again without any problem. What is the exact problem you have?

Heavy Regards, RealHeavyDude.

 

[Cecil]

I've never heard that you cannot connect to a database that uses UTF-8 code page via shared memory..

Little bit of confusion, I mean that you can't connect to a UTF-8 database it just that you shouldn't. Back in version 10.0 I was working on at a company which was using a Unicode database running on Compaq TRU64 and with the help of Tex Texin & Progress Software UK one thing we realized that we could not use CHUI client to do updates to records or even if we export records to disk the Unicode charters would show up as question marks.

It was a common place that we always have to have programs (.p) executed in batch mode to avoid any corruption of any kind. I think we even had a common include file to test if the session was in 'batch' mode, if it wasn't quit.

Example:
mpro database123 -b -p foobar.p -cpinternatl UTF-8 -cpinternal UTF-8 -noutfwarning > theresult.txt

This was the only way we were able to execute program against a progress database. Also we were using RoundTable which would dynamically generate bash scripts and .p programs when we need to apply a schema change. The scripts would call the .p files and apply the modified schema.

Even under Windows we hand custom made OCX objects to emulate fill-in, editor, combo-box widgets built for us to handle Unicode, that was until Progress came out with Unicode compatible form objects (think it was 10.1B).

So that's where I coming from because that what I've always have known. In a nut shell if you try and do and modifications to the schema or records there is a chance that the Unicode characters can become corrupted.

 

[Cecil]

Connect with -S instead of through shared memory?

So if I connected to my database via the socket I can apply a delta schema with out fear of corruption?

Like this??
mpro database -S 10245 -cpinternal UTF-8 -cpstream UTF-8 -noutf8warning

 

[Stefan]

If http://knowledgebase.progress.com/articles/Article/P23910 still applies, connecting with -S will not help. Is there real utf-8 stuff in your dfs? Otherwise I would expect connecting from an iso8859-1 client with -S to work

which is exactly what RHD says

 

[RealHeavyDude]

To shed some light:

Progress always attempts a code page conversion when the two Progress partners ( client vs. database, client vs. AppServer, or, AppServer vs. database ) use different code pages. I deliberately say attempt because these conversion only works when every character in the source code page is present in the target code page. Since UTF-8 supports most characters from most code pages it is almost always possible to convert from a given local code page to UTF-8. Progress provides the convmap.cp file for that purpose - it includes coversion rules between code pages that are compatible. If you attempt an unsupported conversion you will get an error like the conviersion can not be found in the convmap.cp. But, you could still role your own taking into account the you might lose data and Progress will happily apply these conversion rules.

The way back might not be possible without losing characters ( Progress does display them as question marks ) because UTF-8 supports characters that for certain won't be supported by your local code page.
For example you have a client using iso8859-1 retrieving data that was entered by a client using 1250 ( Eastern European, Czech, Polish ). That client won't be able to display all characters that are supported by 1250 and UTF-8 but only the ones which are present in the character set of iso8859-1. Therefore the client will display them as question marks. I think that is the behavior that you describe. Most likely this should not be any issue for df files since - IMHO of course - they should never contain characters outisde the standarad ASCII range. Would you use special characters like é or í in names for tables, indexes, fields or any other database object even if the database would allow you too?

Nevertheless the interactive Progress ChUI client does not support UTF-8. AFAIK that has to do with the fact that UTF-8 is not supported by the terminals in which the ChUI runs - but I am no expert here. Theferore there is no guarantee that you can dump data from a UTF-8 database with the interactive Progress ChUI client - regardless how you connect to the database - without any loss because it simply does not support all characters supported by UTF-8. But, the batch ChUI client ( just like the AppServer ) does. Therefore you can use the batch ChUI client to dump the data from an UTF-8 database without any loss - but, you must use UTF-8 too for the batch ChUI client.

Bottom line: It will be save to use for example an iso8859-1 client to dump & load df files and / or make schema changes - unless ... special characters which are not supported by iso8859-1 are used in database object names.

Heavy Regards, RealHeavyDude.

 

[TomBascom]

The restriction is, IMHO, stupid and should have been fixed a very long time ago.

Also, dumping and loading doesn't involve the terminal -- you do not dump to the screen and then re-key the data... you dump & load to and from files. So even if you do happen to have a terminal that still doesn't support UTF-8 it makes no difference to a d&l.

 

[Cecil]

Thanks for all the detailed information. I feel a bit more relaxed and confident about applying schema changes now. I still need to try out on a dummy database first.

 

[Cecil]

Just a quick update. One thing I've never tried is applying schema changes via the OpenEdge Management Web Interface. Has anyone tried this?

 

[Cringer]

Yes we use it regularly and it works fine.

 

[TomBascom]

I would never use a GUI tool for administration.

 

[Rob Fitzpatrick]

I'm fine with the character tool. It works on any database and can be scripted.