Tech-Blog: Category Linux

vmware broken on Debian Sid

Mike — Mon, 25 Sep 2006 14:43:00 -0400

Just as PB had predicted, my vmware console stopped working after upgrading to the latest Sid. Herewith, a fix, as written by wegface on the vmware community board...

fix is as following:
open /usr/lib/vmware/lib/wrapper-gtk24.sh as root.
search for the lines:
vm_run() {
local exitCode;

add:
# Fix for vmplayer pulling in libdbus-1.so.2 instead of .3
export LD_PRELOAD=/usr/lib/libdbus-1.so.3:$LD_PRELOAD

so your new /usr/lib/vmware/lib/wrapper-gtk24.sh looks like this:
# Run "$binary" while watching its progress on its stderr.
vm_run() {
local exitCode;

# Fix for vmplayer pulling in libdbus-1.so.2 instead of .3
export LD_PRELOAD=/usr/lib/libdbus-1.so.3:$LD_PRELOAD

# Append any libraries that are still missing.
if [ "$VMWARE_USE_SHIPPED_GTK" = 'force' ]; then

now you can have installed libdbus-1-2/3 at the same time and vmware will run, too.

Thanks to Roughnecks from the kanotix linux forum for this fix. I can confirm it works fine

Web quickies

mop — Mon, 01 Nov 2004 18:22:00 -0500

Some tidbits from my Bloglines RSS subscriptions.

Lint4J

"Lint4j ("Lint for Java") is a static Java source code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis."

JotSpot

It’s Wiki++. Typical intranet functionality is available to Wiki users. As seen on John Udell’s blog. (I’ve added John’s blog to my roll.)

Blogging Your Build

Blogs aren’t just for people, you’re processes should be blogging too. Oh yeah.

Debian on Dell Servers

ISOs and pointers for those brave enough to run Dell servers.

AMD64 and Debian

mop — Thu, 26 Aug 2004 12:03:00 -0400

Some notes about my experience installing Debian on an AMD64...

hardware

It’s a refurbished E-Machines T6000 with 512MB RAM and a 160GB Western Digital disk. TigerDirect was clearing them out @ $850. It’s a nice small box, and quiet too.

install

I tried to re-partition the disk to save the pre-installed Windoze OS with no luck. Apparently the Mandrake install disk is able to re-partition XP disks but I was too impatient to burn a Mandrake CD.

I was also too impatient to burn a Debian-AMD64 CD, although this approach is apparently quite mature. Instead, I chose the ’chroot’ approach. Bootstrapping with a normal x86 (backwards compatible CPU platform!) allows the 64-bit OS to be installed in a chroot environment. The debootstrap makes the process almost foolproof. Not quite mike-proof as I was still groking chroots and grub. The biggest mistake I made was failing to mount a proc file system in the chroot dir before switching to the new root.

FWIW, I think it may have been possible to install Debian from CompactFlash, via the USB reader -- maybe in another lifetime.

pure 64 port

The pure-64 port of Debian more-or-less duplicates the packages in Debian Sid. Packages that do not compile for 64-bit are excluded (e.g. OpenOffice), and some packages compile but segfault (e.g. Xprint). I have yet to run XFree or even xfb via VNC, but have heard some success stories. PostgreSQL works as advertised, as does the Blackdown Java port (1.4.2-rc1).

It’s worth noting that the Debian community is still divided about the future of the AMD64 port. The controversy arose because some would like to see the pure-64 port become the official Debian AMD64 port, even though there is no clear way to support 32-bit applications. Bi-arch solutions tend to be klugy, although it seems to be a necessary evil.

performance

So far I can only provide circumstantial evidence about the relative performance of my AMD64 box. It appears to be quite zippy when tasks are in-memory and sluggish when accessing the disk. I need to add some RAM before benchmarking the H database.

specs

albert:/home/mpierson# cat /proc/cpuinfo 
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 4
model name      : AMD Athlon(tm) 64 Processor 3200+
stepping        : 8
cpu MHz         : 2000.162
cache size      : 1024 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext lm 3dnowext 3dnow
bogomips        : 3923.96
TLB size        : 1088 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

mpierson@albert:~$ /usr/local/jdk/bin/java -version
java version "1.4.2-rc1"
Java(TM) 2 Runtime Environment, Standard Edition (build Blackdown-1.4.2-rc1)
Java HotSpot(TM) 64-Bit Server VM (build Blackdown-1.4.2-rc1, mixed mode)

0000:00:00.0 Host bridge: VIA Technologies, Inc. VT8385 [K8T800 AGP] Host Bridge (rev 01)
0000:00:01.0 PCI bridge: VIA Technologies, Inc. VT8237 PCI bridge [K8T800 South]
0000:00:07.0 Communication controller: Conexant HSF 56k HSFi Modem (rev 01)
0000:00:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 80)
0000:00:0f.0 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06)
0000:00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
0000:00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
0000:00:10.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
0000:00:10.3 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
0000:00:10.4 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 86)
0000:00:11.0 ISA bridge: VIA Technologies, Inc. VT8237 ISA bridge [K8T800 South]
0000:00:11.5 Multimedia audio controller: VIA Technologies, Inc. VT8233/A/8235/8237 AC97 Audio Controller (rev 60)
0000:00:12.0 Ethernet controller: VIA Technologies, Inc. VT6102 [Rhine-II] (rev 78)
0000:00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
0000:00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
0000:00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
0000:00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 NorthBridge
0000:01:00.0 VGA compatible controller: ATI Technologies Inc RV350 AP [Radeon 9600]
0000:01:00.1 Display controller: ATI Technologies Inc RV350 AP [Radeon 9600] (Secondary)

albert:/home/mpierson# cat /proc/version 
Linux version 2.6.7-5-amd64-k8 (root@athlon.lowpingbastards.de) (gcc version 3.3.4 (Debian 1:3.3.4-3)) #1 Thu Jul 15 01:13:32 CEST 2004

Converting a MS SQL Server database to PostgreSQL

mop — Wed, 11 Aug 2004 12:46:00 -0400

Herewith some notes from my attempt to migrate a database instance from Microsoft SQL Server 7 to PostgreSQL 7.3. My journey began with Ian Harding’s how-to, and it’s a good place to start.

export from SQL Server

The bcp utility is a quick and flexible command line utility that extracts raw table data (or query result) to a file. It works pretty much as advertised, with the only tricky parts being the treatment of nulls and character encoding. Ian suggested using the -k parameter which forces bcp to use a null character (x00) to represent an empty field, it’s probably the right thing to do. Unfortunately bcp does not distinguish between empty fields(i.e. value is null) and fields containing an empty string. Character encoding can be dealt with in two ways: the -c parameter will force all text data into ASCII text, or the -w parameter will encode text as UTF-16. The two-byte representation would be a no-brainer, except that the data swells to (almost) twice the original size.

Here’s what I used for each table in the database:

 bcp dbname..tablename out ’filename’ -w -k -t "<f-end>" -r "<record-end>" -b 1000

where -b is the number of rows per transaction, and the -t and -r parameters indicate the field and record delimiters. The key when choosing delimiters is to avoid conflicts with field values.

mangle the exported data

Here’s what I did to the export of each table (after moving files to a Linux box):

 # transform to 8 bit encoding
 recode utf-16..utf-8 $1

 # TODO check for literal ’
’, ’	’

 # replace back slash with forward slash
 perl -pi -e ’s!\!/!g’ $1

 # replace tabs with literal ’	’
 perl -pi -e ’s/	/\t/g’ $1
 # replace line breaks with literal ’
’
 perl -pi -e ’s/
/\n/g’ $1

 # replace field delimiter with tabs
 perl -pi -e ’s/<f-end>/	/g’ $1
 # replace record delimiters with line break
 perl -pi -e ’s/<record-end>/
/g’ $1

 # remove Windoze line feeds
 perl -pi -e ’s/
//g’ $1

 # remove nulls
 perl -pi -e ’s/x00//g’ $1

Here’s the step by step explanation:

bcp exports Unicode using UTF-16, PostgreSQL expects UTF-8; and UTF-8 is easier to move around via SCP
the backslash character is significant when importing into PostgreSQL, and I couldn’t think of a reason to keep them in a field value
PostgreSQL uses the backslash to encode tabs and line breaks within fields values
obviously tabs and line breaks are used as delimiters
just housekeeping, I don’t think the line feeds cause a problem
nulls seem to confuse PostgreSQL’s import process

Notes: next time around I’ll use sed instead of perl, but I was too lazy to check the syntax of recode for stream ops; you’ll see that the null characters inserted by bcp to represent empty fields are being stripped - could be that we don’t need the nulls in the exports, or that we should keep them in the export and convince PostgreSQL that they are significant.

create PostgreSQL schema

I used brute force. It would be nice to build a schema.sql script with ant and makedata.

import into PostgreSQL

The COPY command allows table data to be imported from a local file. The only tricky part is the interpretation of null field values; following Ian’s lead I’ve specified the empty string:

 COPY tablename FROM ’filename’ WITH NULL AS ’’;

This approach worked for all tables except those that contained empty strings in columns defined as ’NOT NULL’. I kluged these tables by altering the schema: "... ALTER COLUMN xxx DROP NOT NULL".

update

It’s probably also a good idea to run the maintenance.sql script to clean up some tables before extracting. Smaller is better.

PostgreSQL benchmarks

mop — Tue, 29 Jun 2004 21:59:00 -0400

PostgreSQL has been the highest profile open source transactional RDBMS for a while, deserved or no. It gets the most eyeballs and one hopes that this translates to reliability and performance. My experience with a certain proprietary RDBMS and PostgreSQL suggests that open source translates to reliability. But performance? Don’t know.

Benchmarks have always been tightly controlled by the big vendors like Oracle and Microsoft. Can’t blame them for discouraging the publication of ad-hoc test results -- it’s a non-linear system and results are highly dependent on conditions beyond the control of the vendor. The Transaction Processing Performance Council (TPC) is a non-profit that publishes benchmark results from standardized tests. You’ll find some impressive numbers for Microsoft SQL Server, IBM DB2, and Oracle, mostly running on expensive hardware. So where are the open source benchmarks? I’m guessing that nobody has yet ponied up the cash required to put PostgreSQL through the TPC tests.

Cheers to the Open Source Development Labs for taking on the task of creating a test suite and providing an alternative to the TPC. Their test suite currently runs with PostgreSQL and SAP DB (now MaxDB). Some of the results are un-impressive, 20 TPS on 4xPIII 700MHz, when compared with commercial RDBMS. More interesting are the evaluations of the Linux kernels and file systems.