Cannot generate Unicode data correctly by DTM Data Generator

Powerful and fully customizable utility that generates data for database testing purposes

Moderator: Support Team

Cannot generate Unicode data correctly by DTM Data Generator

Postby Trial user » Tue Mar 03, 2015 10:53 am

Hi,

I have downloaded the demo of DTM Data Generator Enterprise (Win32, Unicode, 1.55.01) version for evaluating, the tool is great and flexiable to handle complex cases that fulfills my needs. But I encountered problem while generating Chinese and Portuguese value, so as using the FirstName_National from Value Library. Please find my cases:

1. Using FirstName_National from Value Library, the data is messed up after inserted in the database table:
Gerald
C?cilia
Berthold
�`�l�
Else
Ciriaco
Cipriano
Ad?n
Pierre?
�h�\
�j�d�\


2. Select fields contain Chinese value and Portuguese value from SQL console in the DTM Data Generator
ABCV~OOOEEÒÈ becomes ABCV~OOOEE??
陳大文 becomes ?喳之

3. In the Data Rule, set the fill method of a column as SQL statement "select chinese_name from table_a"
The value in the generated rows of this column are messed up, it should be Chinese value.
I have checked the generated text file which contains the INSERT statements and it looks fine, but the data of this column inserted in database table is messed up.


I am connecting to an Oracle database 11gR2 with unicode setting through ODBC.
My PC is Windows 7 under Wow64.

May I know is there any setting of the tool or my PC could solve this problem?

Thank you~~!
Trial user
 

Re: Cannot generate Unicode data correctly by DTM Data Gener

Postby Support Team » Sat Mar 07, 2015 9:52 am

Please be sure that:
1) you have NVARCHAR2 data type for target column, not VARCHAR2. The first one supports UTF16 correctly.
2) you have correct NLS_NCHAR_CHARACTERSET option specified for UTF16. Otherwise Oracle can't recognize INSERT statements with Unicode correctly.

Also, if you have external data files, all of the must be Unicode as well.
Support Team
 
Posts: 257
Joined: Sun Mar 26, 2006 3:48 pm

Re: Cannot generate Unicode data correctly by DTM Data Gener

Postby Trial user » Wed Mar 11, 2015 4:48 am

Hi, thanks for your reply.

This is the setting of my database
NLS_CHARACTERSET UTF8
NLS_NCHAR_CHARACTERSET AL16UTF16

1) For my exsisting table, the fields are using VARCHAR2. So, I added a NVARCHAR2 field to test.
If I update the this field by SQL UPDATE statement in Epilogue SQL field, the result if fine.
If I set this column in the 'Columns' tab in Data Rule by Value List with English values, the single quote for string is missing. i.e. TEST_NAME=PETER but not TEST_NAME='PETER'. Does this type is not supported?

2) If I don't use NVARCHAR2 but VARCHAR2 (using UTF8 according to our setting), could I set something of my PC in order to generate data correctly for Chinese, Portuguese, etc...? I tried to set the NLS_LANG in my PC as AMERICAN_AMERICA.UTF8 in registry and also added as system environment. Seems it does not work. Need to install a newer ODBC driver? As only 32bit version of the data generator is available for trial, I need to update my 32bit ODBC in my Windows? The 64bit one on my PC is updated according to my installed Oracle Instant Client 11g. Will it help?

Thank you very much~!!
Trial user
 

Re: Cannot generate Unicode data correctly by DTM Data Gener

Postby Support Team » Thu Mar 12, 2015 9:17 am

1) For the first case we need complete CRETE TABLE statement and exact Oracle version to analyze it.

2) unfortunately the generator does not support UFT8 at the moment. You can use ANSI or UTF16 only.
I hope we'll be able to do that in the future updates.
Support Team
 
Posts: 257
Joined: Sun Mar 26, 2006 3:48 pm

Re: Cannot generate Unicode data correctly by DTM Data Gener

Postby Trial user » Thu Mar 12, 2015 10:59 am

1) As we are not using NVARCHAR2 actually, it's OK to ignore this issue at the moment.

2) I see, thank you very much for your replies and hope it will support in the future.
Trial user
 


Return to DTM Data Generator, Flat File Generator and Test XML Generator

Who is online

Users browsing this forum: No registered users and 1 guest

cron