Unix Admin Corner

Node on Nexenta

2011-08-16T22:31:00.001-05:00

I discovered node.js that Joyent is using extensively to work its cloud magic. What is node.js check out this video for info, it seems very cool.

http://www.youtube.com/watch?v=jo_B4LTHi3I&feature=player_embedded#at=2825

Now the next question becomes how do I get it running on Nexenta. First hope was that it was already there considering the following announcement of the of the Joyent/Nexenta partnership

https://www.nexenta.com/corp/newsflashes/102-2011/918-joyent-and-nexenta-partnership

jamesd@amd:~/nodejs/node$ apt-cache search node | grep ^node

jamesd@amd:~/nodejs/node$

nope no luck, okay that was out, well since I wanted to give it a try I popped over to my Ubuntu box and then built node using http://www.codediesel.com/linux/installing-node-js-on-ubuntu-10-04/ and it worked fine, and was painless. Except the git node repository has moved to github.com/joyent/node.git

opteron:nodejs$ uname -av

Linux opteron.themagic.com 2.6.32-33-generic-pae #71-Ubuntu SMP Wed Jul 20 18:46:41 UTC 2011 i686 GNU/Linux

opteron:nodejs$ node

> process.platform

'linux'

Well now for a leap of faith... back to Nexenta and give it a try.

To make a short story even shorter, it WORKS!!! using eveything just the same.

jamesd@amd:~/nodejs/node$ uname -av

SunOS amd 5.11 NexentaOS_134f i86pc i386 i86pc Solaris

jamesd@amd:~/nodejs/node$ node

> process.platform

'sunos'

Moving onto Nexenta Community edition

2011-05-04T23:07:00.002-05:00

A while ago I manged to screw up my ZFS pool by having it use files on other ZFS pools as cache and slog devices. After 6 different versions/distrobution/releases, I have given up, and recreated pools that are much saner in layout.

The filesystem now runs Nexenta community edition

With the following pools
250GB root pool or syspool as Nexenta calls it
3x 500GB in a mirrored layout in a pool called tank I may break it into a 2 way mirror later. And then create another 2x 500GB pool or zdev on the pool, currently this pool is mostly being used for ESX nfs storage. I wanted to go 3 drive wide mirror for read speed and improved write performance over raidz.

I was going to use SXCE on the system but during the install it hung at 18%, so I gave up and went with Nexenta since I had the disc sitting next to the machine anyway. I am enjoying it mostly did have to add another repository to get bind9 build for it, but that plus about 15 minutes of configuring it I was able to get nexenta to be a DNS server, thus making NFS happy.

Future:

Well I don’t expect to be using this fileserver long, since I want to move to a all in one box solution for ESXi+ZFS using an HP ML350g6, but this will allow me to add more storage to my existing ESXi 3.5 box, its built in 60GB is showing its age with its SCSI drives that are expensive for anything larger than 36GB and use lots of power in return. I will be picking up components to go in the ml350 that can live in this box till the main (expensive) bits are ready.
I will probably get 4x 2TB drives and put them in this box perhaps even the ssd’s for l2arc/slog can go in this box as well.

Success!!!

2011-04-17T11:53:00.002-05:00

Well after 4 days of struggles and unexpected patching sessions, I now have my new 750GB SATA drive in my laptop. Once I started using samba it all went much better than I expected. To make things faster be sure to choose restore partition not files. Restoring files takes a lot more time. Despite your first guess that restoring by partition doesn’t require extra steps if you installed a larger drive, it uses the partition scheme you setup on your new drive. You just need to specify where you want your files restored too. The process goes much faster, when I tried using the file method it gave me a time of 19 hours, and went up occasionally to 3-5 days. When I switched over to partitions in the morning when the timer still gave an ETA of 18 hours, it went down to 2 hours, but took like 3.5 hours to complete the process.

When I rebooted the laptop after the restore completed, it immediately started booting Windows, no hiccups, no little tweaks, it all just worked, I was surprised and impressed at how well it did the job. Of course this raises the question could I have used full drive backup and got better performance, and still restored to a larger partition?

After windows booted, it did have to look for and install a driver for the hard disk, I didn’t change the controller, so I was surprised it needed to install a driver for a SATA hard drive that replaced a SATA hard disk? But oh well it worked, I will have to see next time I reboot if it gives any performance enhancement.

The reason I purchased Acronis and didn’t use Clonezilla and gparted type solution was because I was afraid that I wouldn’t be able to make Vista boot on the new disk, all the entire how-tos I read required repair work and/or recover disks. My laptop didn’t come with a Vista install disc; I did create the recovery disks nearly 3 years ago, damn if I know where those disks are hiding. So in the end Acronis did what I expected just the backup progress wasn’t as easy as I expected, but the restore pretty much exceeded my expectations by far.

But it is nice to see 487GB of free space, and even gave 60GB of space to Linux partition should I choose to dual boot sometime in the future.

Lessons learned in this process.

Install as Administrator, an administrator enabled account isn’t enough, after the restore process it still doesn’t like running as a normal user.
Don’t bother using FTP with Acronis it really doesn’t play well with others.
Defragment your drive before you start, backups go faster with it defragged.
Use the fastest transport media possible, wired is much better than WIFI even “N” class wireless
Gigabit would of helped, but wasn’t an option on this laptop
Don’t use dynamic style disks the restore process doesn’t like it, and you can change later
Be sure to have another computer if possible for several days while the backup/restore is proceeding.
Install gkrellm on your fileserver so you can see throughput, and be sure that Acronis is doing something.

Progress

2011-04-16T22:30:00.002-05:00

I was never able to make Acronis Trueimage 2011 complete a full backup to any Linux based FTP server, I finally just found the simplest samba configuration document and created a samba share, which worked painlessly. I even took the extra couple hours for it to complete a full validation of the backup before beginning the drive replacement procedure. Not sure if I read something for an older version about not using samba for the backup or what, it was completely painless except for the fact that it took all the way to Sunday to complete a full backup of a 180GB disk. Something should have been done in 8 hours.

I bought the Acronis Disk Manager Home along with the Trueimage home, it was about $65 combined for the together not bad, it made the job of formating my new 750GB 2.5” SATA disk go smoothly and allowed me to setup a Linux partitions for ext3 and swap. BTW if you are in the market for laptop hard drives, the current 1TB drives are 12mm in height and most laptops can only take a 9mm tall drive so don't order the 1TB drive that you really think you need before you verify it will fit in your laptop and save your self the return and re-order hassle.

I sit here and type this as the recover operation runs so we shall see how this all turns out in a few hours. Its only reading data off the samba server at about 7-8MB/s compared to 12MB/s that it wrote at. I guess Linux based recover program is slower than the window hard disk based version. I'm pretty sure the new hard disk is faster than the old one it replaced and the 1.5TB disk in the Sun Ultra 20 with dual core 2.6ghz Opteron should be up to the task of filling a 100mbit ethernet to the laptop, the ulra 20 has a Gigabit Ethernet port and is plugged into my gigabit wifi router.

The process has been going on for about 30-40minutes, and it still hasn't provided any hints as to how long the recover process is going to take, thankful I have gkrellm running on the Ultra 20 so I can see that it is sending between 1.5 and 6MB/s across the network. It still doesn't appear to have any color to the progress bar on the Acronis recovery dialog box.

I guess I'm going to take my own advice in these situations, walk away and let the computer do what it does best.

UPDATE: well after about an hour, it did give me an ETA for completion, I really don't like the number, 19, 19 hours but what can I do. But its the story of this progress everything takes much longer than I think it should!

This Close!!!

2011-04-16T00:02:00.000-05:00

I decided to buy TrueImage 2011 home, to backup my laptop, since I will be replacing its hard disk, I am seriously thinking of returning the software and asking for a refund, it’s been a total pain, first issue had to do with permissions and it not liking the software I had installed previously not sure which software, but I had to disable my virus checking software, and every service not written by Microsoft, and then run it as “Administrator” not sure which one of those things killed the initial startup, but I did get it going finally, But that is only the first stumbling block.

I am using the ftp destination for backing up, which basically goes like, first time I tried it with just plain wireless, N class network adapter and after it uploaded its first gigabyte it estimates the ETA of 2 days and 8 hours. That would never do. So I hook up wire and see if it could recover, it can’t FAIL. Like really it can’t restart a ftp connection? How lame.

But it gets better. I restart the backup process, this time it gives a better speed only 8 hours to backup the 160GB hard drive. Well it’s now about 1am, and I have to work in the morning, so I lock my screen and head off to bed. I wake up in the morning thinking it will be done or perhaps I will have an hour or two left. You guessed it, it’s not done, I sit looking at my login screen, Microsoft decided it was the right night to install patches, and reboot my machine around 2am…. Not sure how much I blame Acronis for this issue, but it could of made Microsoft wait… I kick the backup off again and head off to work. I get home, to find an error message and the backup has failed, used their trouble shooting tools that link into their knowledge base, a few hits, but nothing seems valid, so I try again, and 8 hours more, FAIL again. Lots of fighting with ftp connections, sure you can create them, but they don’t give you a way to delete them and start over, talking about second class destination.

Well this process started Wednesday at around 6pm, it is now Friday 11:39pm, I finally got tired of fighting this and decided to open a ticket, I avoided this process because it feels too much like being at work opening tickets, of course there automated system decides to use Microsoft outlook to send off an email, I have outlook installed but not configured so fail again. I think spent another 20 minutes looking for the way to upload it manually. After I upload there “troubleshooting packet” the upload process tells me it knows a possible solution, so I read the first link and low and behold comes a page that mentions it doesn’t like proftpd, that I have been trying to use, okay you know your software hates proftpd and needs features that some ftp servers don’t support why not check for them and bitch about the lack of them preferably in pure English FAIL again. Further the doc doesn’t mention a ftp server that they support FAIL again. If you Google Acronis ftp server, you get there software to backup Linux servers. So we have a company that supports Linux, but won’t give you a list of ftpd servers that works in Linux with Acronis trueimage and I couldn’t even find a list of supported ftp servers on their forum. It seems like there whole error reporting is for machines, every error is a hex code, and can’t you print out the error means in English. This is a home product…

For those that found this blog entry because you have been bitten by the ftp problem. I did find an ftp server that appears to work, it is pure-ftpd which ubuntu has, all you have to do is install it.

All in One?

2011-04-09T20:19:00.000-05:00

Well the debate continues in an earlier blog post I discussed my thoughts on my new servers for fileserver and esxi box (debating new home hardware) my current thought has now changed from getting two servers into just getting one but running Solaris ZFS in a guest on it. I know ZFS loves memory, so to make Solaris or Nexenta ZFS perform its best I would get one HP ml350 g6 but upgrade it into the middle of its capacity, 2x quad core e5606 (3.13 ghz 4MB of l3 cache), and 4GB of memory from hp, (2x 2GB dimms from hp) and then 4x 8GB dimms from a 3^rd party memory supplier for a total of 36GB memory. For the controller I will use the p410/nz and grab a 512MB cache with battery backup to get full write performance for the disks I will add 2x 250GB from HP to hold Solaris and a couple other guests, and 4x 2TB sata drives from a 3^rd party in a raid 1+0 layout, two 460 watt psus. Even with the extra 8GB of memory and extra power supply it works out to be $800 less and more importantly it also uses less power so lower electric bill long term.

I think the above configuration should be a good enough performance. But if it still needs more performance Napp-it all in one, documents using a SAS controller in pci pass through mode with good results. The card they recommend is LSI 1086E SAS controller that Amazon has on sale for $160 but then I may need to get a separate disk box or put the disks in the non hot-swap bays. I don’t think my storage needs require overly fast performance and I could easily dedicate 8-12GB of memory to the Solaris/Nexenta guest perhaps even more. Third party 8GB dimms are only $270 and the ml350 g6 has 18 dimm slots so its not like I’m going to run out or I could possibly get a SSD device and attach to the p410 controller and pass that into Solaris/Nexenta guest.

Loving Autohotkey

2011-04-09T09:34:00.000-05:00

As a Unix administrator, I try and use my keyboard more than my mouse, and I love automating things and processes, until I found Autohotkey I really didn’t have a good way to do it in windows, sure you can record marcos or write small scripts in office basic or whatever they call it now, but that is limiting, or I could write scripts in virtualbasic , too much work.
Autohotkey really makes automating things like passwd entry, I know it’s a “bad thing” but when you are a new job site, and they really don’t understand the easy, power, security of ssh shared keys but they accept autohotkey what can you do? When in rome I say.

It would be nice if Autohotkey had a tool that allowed you to encrypt a string decrypt it easily so that you wouldn’t have to put passwords unencrypted in your script files, sure someone that really wanted access could crack them but it would at least keep the casual person from reading all my passwords.

Something like
myPasswd=”#$@SSGFSRWGSsses”
Send unencrypt(“%myPasswd”)

Here is my current autokey script on my home system.

#s::
InputBox, SearchTerm, Search
if SearchTerm <> ""
Run chrome.exe http://google.com/search?q=%SearchTerm%&ie=utf-8
return

#w::run winword.exe
#e::run excel.exe
#m::run chrome.exe https://mail.google.com/mail

Music Cloud

2011-04-06T07:25:00.000-05:00

If you haven't heard amazon won the race to the cloud for music, you start with 5GB for free download any album and get up graded to 20GB, and they dont count amazon purchases towards your quota. Now to get you started they have a a few good deals including four 99 mp3 albums for 99cents. I grabbed those four albums and motley crue's greatest hits for $3.99, now I can have music at work without having to remember to bring my mp3 player everyday. It so far works in google chrome, and will try internet explorer and firefox at work today.

Seed your cloud

Audio Geeks should check this out.

2011-03-16T09:30:00.000-05:00

audio geeks out there should check out http://www.vinylphilemag.com/pdf/vinylphile-005.pdf its a free publication done by a friend of mine, Rich Teer, it comes out monthly even if you don't care for vinyl its a worthwhile read for the audio gear stories.

Debating new home hardware

2011-03-15T13:55:00.001-05:00

Thinking about buying a couple new servers for home, a new ESXi 4.1 server and aSolaris ZFS fileserver, Here is what I decided the requirements of the first server are.

ESXi server requirements
Fully supported on the VMware HCL
Dual socket, quad cores 4MB l2 caches per proc
8+GB of ram, system needs to support at least 32GB for future expandsionHardware raid controller (makes esxi happier)
At least 4x 3.5” sata drives (may consider 2.5” drives)
Dual gigabit Ethernet port.
3 years next day business hours hardware support
Out of band management nice to have

I have been doing computers for almost 30 years in one form or another building x86 gear since 1985, and frankly I’m tired of working on my home hardware, I want something that is pretty much plugNplay I don’t want to mess with the hardware and I don’t want to have to worry about replacing the system for 3 years by that point there should be some more interesting things going on. I am leaning towards a standard tower server they tend to be quieter than there rack mount cousins and don’t feel the need to pay the rack mount tax as well. I currently have a dl380 g3 that is very loud and has no slow fan speed mode at least in ESXi 3.5. The current sweet spot for cpus are quad core 2.13ghz, slower only saves $20 and faster is a price jump of over $100 per cpu. Upgrading my current ESX server will also allow me to run 64bit OSes and will give me the excuse to power down another server as well for the most part.

I looked at Supermicro systems but I don’t see much price difference between them and HP that offers everything from the system to full support perhaps I am not finding the right systems. Supermicro offers some systems with lots and lots of disks, which I don’t need for home. Also lacks support and enterprise out of band management features of HP.

I also looked at Dell but perhaps I didn’t find the comparable model but I really didn’t find a lower price for my configuration, and Dell seems to be a slightly less of a reputation with less enterprise quality add-ons so less stuff to play with. So Dell is out of the running.

I looked at Sun/Oracle x86 options but they seem to be limited to rackmount gear that only supports 2.5” sas drives and the prices were way more than I am looking at and there online configuration tool is offline so I couldn’t reduce the ram and drive size to see if that helps the price. I really want to go with a tower based system because they are quieter. Cheapest x86 server I can find is over $3000, almost double what I have budgeted.

The purely beige box solution seems appealing because of price but it is hard to find a website that offers complete configuration of systems with dual sockets and they will also lack high quality support and the enterprise out of band management features. I really don’t want to assemble my own hardware.
So far my choices for the ESXi server are either HP Proliant ml150 g6 or ml350 g6 I am leaning towards the ml350 because it comes out of the box with a 3year warranty and it doesn’t require extra licenses to support the same configurations that I may upgrade to in the future it’s a bit more but you can expand it further. With the ml150 I need to get the hot plug model so I get a raid controller that works with ESXi out of box, the low end integrated controller isn’t supported by ESXi.

The shocking things with my configuration at the HP website is that the ml350 is cheaper than the ml150, among other differences the ml350g5 supports both the e5606 and e5506 cpus as opposed to just the e5506 that the ml150 g6 supports, the e5606 is cheaper. The specs are the same for these two cpus, same speed, same l2 cache, same watts, after looking at the Intel site I see the e5506 uses 32nm press as opposed to the 45nm that the e5606 uses. Which I see on various websites not to add anything except slightly lower power usage, the e5606 is fine for me. The low power cpus are too expensive I don’t think I would save enough to make them a reasonable choice. I keep debating on whether to wait on the second cpu, I like the idea to buy it day one so that it comes installed and fully tested, I’m sure it will add some speed and memory expandability but I won’t be using all 8 cores for a while. I never really found dual 3.2 GHz p4 in my old esxi server to be limiting it was always about memory.

Adding the redundant hot swap fans seem to be a good choice, an extra $50 for security having a fan fail while at work or while away on an extended vacation which is the time most hardware seems to fail. Would lead to a shortened life span of the server perhaps keeping it cooler all along I’m not going to keep the server in an air conditioned space it will be in my basement that may see temperatures in the 80’s perhaps even 90’s a few days a year.

Both servers support more than enough drives I will be purchasing one drive from HP for ESXi probably a 250GB sata drive $90, the extra $20 compared the 160GB drives is worth it, Larger drives from HP are too expensive so I will be adding standard home grade drives, and buying enough empty drive spuds on ebay to add 3 more disks, a second 250GB for mirroring of the primary drive and then 2x 2TB SATA drives for storage of guests all configured in mirrored layout for data protection and my low usage level should make the higher enterprise drives unnecessary. These later drives may be added later. Another downfall of the ml150g6 is that if I add the second drive cage increasing the internal compacity to 8 drives from 4, I would need to upgrade to the 750watt psu a $175 upgrade plus the enablement kit a $200 increase which seems really excessive and pretty much puts the final nail in the ml150g6 configuration in this comparison. I don’t see a need for all 8x drives but I do like the option of doing it later. I have thought about going with sas drives but they are a bit more expensive for 3^rd party drives and way more expensive from HP and are limited to 750GB currently. Frankly I don’t think I will see much speed difference because I won’t be pushing them hard with my work load, how much performance do 5 part time users need, and I can add more cache memory to the controller as well.

I would most likely be ordering the server with 2x 2GB dims for memory one dimm on each cpu as required by the hp website again the extra $20 between the 1 and 2GB dimms is worth it, and will be adding more 4GB dims from a 3^rd party memory supplier, they are about $100 each and ½ the price of HP’s memory and I can remove them should it become an issue for service personel.

I will only be purchasing one 460watt power supply and possibly adding the second power supply off eBay later for redundancy.

Dual gigabit Ethernet is important because I can allocate one port to iscsi/nfs IO and the other used for regular network traffic . The ml350 has dual gigabit standard, the ml150 would need a pci-e card. Long term I may even decide to add a 3^rd gigabit port the cards are cheap and getting cheaper by the day.

I also want hardware support for 3 years, by then I will be ready to replace the server, if I am not it should be at the point the where parts are dirt cheap and the parts should be easy to find no reason to pay more for a longer contract. The ml350 includes 3 years support, on the ml150 it’s a $188 dollar add-on. Adding the 4^th year increases the price $280 which just too much for a server that old.

So in the end the ml350 wins because its nearly the same price out of box and then all the extras add up to making the ml150 more expensive and less expandable in the end.

Server two, Solaris ZFS server

Dual socket (because expandability is nice) but only one cpu installed

6x 3.5” drives, 2x small drives for OS and 4x 3^rd party 2TB drives in raid 1+0 for a total usable of 4TB before compression

SSD for l2arc and slog 80GB added sometime in the future.

Dual gigabit or more
3 year support.

This server comes out a bit less since it only has one cpu, quad core because dual core options don’t save much money anyway.

I will buy with only 4GB on board memory and then add 3-4 more 4GB 3^rd party memory mostly used for ZFS caching.

250GB sata drive purchased with the system, with 1 more 250GB later and 4x 2TB drives added for home storage, less than $100 for 50% more security and increased performance I will also try to buy from 2 different drive vendors for a slight increase of potential security because less chances to up with a bad drive lot taking out the zpool.

All the other arguments remain the same.

Now the question I have is, how much of a discount can I get if I purchase from a HP reseller, my two servers I will not be enough to get me a big discount but a 5% discount would be nice. Anyone know of HP reseller that give discounts and preferably local to the Milwaukee, WI area.

why not to become a netadmin

2011-03-10T19:11:00.000-06:00

Every Network admin's nightmare 5days of no sleep and having to revert back to paper for all the major actions of the company.

Now this is one major torrent

2010-10-28T23:00:00.000-05:00

900GB of ancient web content, might be interesting to see, not sure I would want to give up that much space to do it, but it is only 1/2 a modern 2TB. I think some stuff I posted somewhere ended up being posted on a geocities site, so may be interested, maybe google will archive it and make money on it, and show yahoo how ads are profitable even if yahoo diddn't want to devote the servers to it.

Status update, gave up with the process.

2010-10-25T21:58:00.002-05:00

Well after over 3weeks of no progress, I just gave up, and powered down the system, I will give it another try when a new version of OpenIndiana is released. At that time I will setup for a long process, enable ssh, and possibly screen so that I can run multiple processes. Of course with my luck it will take 5 minutes ;-)

Perhaps lost in a corner

2010-10-13T00:26:00.002-05:00

Well I still let the import process continue, not sure if its proceeding, but the disk access led continues to blink, and blink I hope it will finish sometime.. I will have to set an end, but I haven’t yet.

To kill or not to kill

2010-10-04T10:16:00.000-05:00

About 2 weeks ago I attempted to import a pool (4x 500GB SATA drives) with zvol based slogs devices that were having issues in b134 after repeated hard crashes due to power issues. This attempt was done using a live Open Indiana DVD, after I imported the first two pools on the system I executed the import of the pool that I hoped would complete successfully this time. Even though It failed with the older versions.

Well not knowing or expecting that this import would take more than an hour or two, perhaps a day, I didn’t do anything to prepare the system to be in a state that would allow for monitoring theimport process. It was booted into text console as opposed to the default X gui based console, and sshd was not started. Thus I had no way to monitor the system that was stuck in the kernel, it appears to be progressing the disk activity was sporadic, barely noticeable only by the blink of the amber LED, some blinks were just a quick flash occasionally some were a flash that lasted a bit longer sometimes they occurred every couple seconds, some time they go 5 or 6 seconds between brief flashes but since the system was running on a live image with no writes or reads to hard disks to light up the disk access LED so it obviously working on the command I gave it, I had even pressed ctrl-z in hopes of pausing the command in a way that it could restarted in the background later and giving me the chance to run a command or two to monitor the system progress or at least what is happening going on.

But two weeks later the process continues only giving me two options, let it continue trying to work its magic and hopefully end successfully and giving me the chance to remote the zvol based slog devices from the pool or even just get a copy of the contents of the pool stored on valid pool or a different system with the required space. Or do I give up and shut down the box only to start it a way that would give me the chance to monitor the process and/or remove some zvol’s that aren’t par t of the pool hopefully speeding the process the next time in hopes of speeding the process.

I guess I will let it go for a few more days in hope that it will at least show some sign of completing shortly,

OpenIndiana Update

2010-09-25T08:42:00.001-05:00

I have ran up to ONNV b145 on this system, and was able to run “zpool import” scan without issue,but for some reason a single drive in the pool was causing it to crash with OpenIndiana b147. It didn't matter if the drive was on the on-board controller or the added realtek controller. Just the existence of the drive caused the crash, I didn't notice any scsi errors. And I had not recently.

Some background on why I want to move to OpenIndiana a few versions ago, I heard that you could remove slog devices, so I decided it was safe enough to use a zvol from another pool (one made up of a single 1TB drive) as a slog device making the raidz pool much more usable for nfs based vmware ESXi storage, what I didn't relive at the time was that yes you can remove a slog device from an imported pool, but should the slog device die or the underlying zvol get wedged somehow it wasn't possible to import the pool. Of course a weeks after adding the zvol based slog device the pool refused to import after a lightning storm came through one evening and caused power to fail 5 or 6 times in a couple hours.The zvol became wedged, can't snapshot or clone the zvol or use it there may be fixes for this in post b145 builds.

Now as of b146 it is possible to import a pool with missing slog devices. This should allow me to get my 1.5TB pool of mp3's, movies along with all the data files back on line. If you choose to follow my bad example and use a zvol as a slog device, I dont recommend it, the key to importing the pool is.

zpool import -m -f -d /dev/dsk/ -d /dev/zvol/dsk/[slog zvol pool]/   poolname    

Dont expect the above command to finish quicky, I started it around 10 hours, ago and it hasn't returned though I still see infrequent disk access on the system running openindiana live dvd. As soon as it returns hopefully successfully I will be backing up important data, and probably recreating the pool without the zvol slog devices.

OpenIndiana

2010-09-24T13:49:00.002-05:00

Tried OpenIndiana over the last few days, well I tried too anyway, first attempt was using there “pkg” enabled upgrade process, but that didn’t work because somewhere along the like I fubar’d something(s) over the years, even went back to my initial boot environment and gave that a try it failed as well.

Next came the USB drive method, the write process seemed to go well, but when booted I got a simple message on the screen “GRUB” nothing else no chance to edit or change anything. No boot happened the box was hung.

So decided to give the Live DVD a try, it burned and booted as expected, but then the real test came, importing my zpools, well I typed pfexec zpool import, then the disk light came on and 2 seconds later the box crashed and rebooted. I tried all the grub choices but still no luck.

This is a pretty run of the meal beige box, asus m2a-vm amd x5200+ dual core cpu and 6GB of ram, it ran b134 and b144 fine no issues, b147 doesn’t like it, I do have a realtek sata -> pci board in it that I will try removing later and see if that is causing the issue but that is a project for another day.

Checking Dolly’s Paternity

2010-06-22T09:50:00.001-05:00

Here is a one liner to print out which file systems and zvols in your ZFS pools are clones and print out its parent.


zfs list -s origin -r -o name,used,refer,origin  | sed -n '/-$/d;p'

Steps to Clone a VMWARE guest on a ZFS NFS share

2010-05-26T19:59:00.000-05:00

Today’s task was to create a base image to use for Windows on my ESXi box so I don’t have to sit through windows install everytime I want to play with windows apps. Of course being a starter ESXi system I don’t have vCenter or anything else that provides vMotion. Here are the steps.

Create a ZFS filesystem and share it with root read/write permissions.
Set your favorite zfs options, disable atime, enable compression, etc.
Setup nfs sharing with root privs

zfs set sharenfs=root=@192.168.1/24 tank/esx2 # please make as secure as you wish.
zfs set quota=20g tank/esx2
# unless your cpu is really slow I think compression would be great because all of these
#blocks will be shared among the clones
zfs set compression=on tank/esx2
zfs set atime=no tank/esx2

Go to your ESX(i) box and create a new dataset using the newly created share.

Create a new guest storing everything on the new datastore.

First I installed 2k3 enterprise installed 16GB harddisk ( yes way too big, but if you don’t use the space its basically free, and it’s a pain to run out).

Then I installed all my favorite applications and plugins so I only have to do it once.

Vmware tools
Filezilla
Firefox
Isomagic
Putty
Adobe reader
SNMP ( nice to add this so it’s easier to monitor)
Flash Player
Notepad++

Now configure a few things, so we do it once and don’t have to do it again.
Set hostname to be “baseimage” or something.
Make sure the guest is set to use dhcp so you don’t have conflicts later

Create user accounts
Configure proxy
Configure web homepage
Configure snmp
Configure ntp

the next steps I’m not sure how necessary they are but I figure I would do them anyway.

Clean up C: drive
DEFRAG DISK

Now poweroff the guest

Login to your OpenSolaris box, take a snapshot of the guest filesystem

zfs snapshot tank/esx2@base_windows_image
zfs clone tank/esx2@base_windows_image tank/new_guest
zfs set sharenfs=root=@192.168.1/24

rename the directory that holds your base image to something related to your next guest, changing the directory name won’t use more than 1 block of space unless you give it a really long name.

Now back to the esxi box and add another datastore with the cloned directory

Create a new guest using custom method, use an existing image
Choose the one on the freshly cloned filesystem
Add/remove hardware as you desire, does anyone use floppy disks anymore?

Boot it up everything should work

Now reset the hostname to what you need it to be.
Give it a static ip if desired.

Customize the guest as you see fit…

Done

The ZFS pipe is clogged

2010-05-23T19:56:00.000-05:00

Here are the changes that are stuck in the ON pipeline waiting for Stable OpenSolaris 2010 to get out the door. A lot of performance improvements and usablility stuff, I'm sure I missed a few.

b135
zio_ddt_free is single threaded with performance impact
zil need not inflate blocksize that much
slog can probably pack 2X more data per lwb
dsl_sync_task_group_wait() can wait too long

b136
old iscsi target dies...
BUG/RFE:6922272SATA framework does not handle greater that 2TiB disks

b137
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6917066

b138
For 4k sector support, ZFS needs to use DKIOCGMEDIAINFOEXT

b140
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6280630
add interval (count) args to zpool list
dbuf_read_done() locking performance improvement
RFE: provide interval arg to zpool status to monitor resilvering
COMSTAR should avoid extra data copy to zvol-based backing store

DeDup will get faster on lowend machines

2010-05-23T18:46:00.001-05:00

At the end of Dedup Status thread is a cool little tidbit that I missed but it gives all us Home OpenSolaris users that can't afford expensive SSD's something to wait for, It is putback in b141, and as that the wait for the next OpenSolaris stable has put all /dev releases on hold we all get to wait, hopefully.. they jump a few releases when stable opensolaris is released so we don't have to wait 3 months to get this fix. There are bunch of other fixes that went into OpenSolaris /dev that the non hard core guys that don't build there own releaes have to wait to get as well.

George Wilson wrote:

I just integrated a performance improvement for dedup which will
dramatically help when the dedup table does not fit in memory. For more
details take a look at:

6938089 dedup-induced latency causes FC initiator logouts/FC port resets

This will improve performance for such tasks as rm-ing files in a dedup
enabled dataset, and destroying a dedup enabled dataset. It's still a
best practice to size your system accordingly such that the dedup table
can stay resident in the ARC or L2ARC.

- George

Feeling Old yet?

2010-05-23T12:13:00.001-05:00

PAC-MAN turned 30 this week.... Spent hours playing this game as a teenager, had it on my Atari 2600 and played at the arcade...

VMware Lab updates

2010-05-18T21:12:00.000-05:00

My newest gear, a HP DL380 G3, is doing great I have managed to create 4 guests for it 3 of them are running Windows 2k3 server. Its making a great test ground for new applications including HP SIM 6(Insight Manager) yes it's a pig it needs 2.5GB of ram to install smoothly with MS SQL express on board. An old copy of Citrix Meta frame I had laying around which is coming in handy since VMWare infrastructure client is a windows only application, and my Windows laptop is out for repairs. And my main desktop is now a Sunray running off a blade 1500. So I added VMware Infrastructure client as an application and installed the Solaris Citrix clients and it works very smoothly. I tried installing the Citrix client on opensolaris, but even after adding the necessary libraries it just kept exiting with an out of memory error, and the box has 5GB of ram and 7GB swap partition so I don't think that is the real reason, perhaps I will look into the issue more later.

GNS3 rocks but I think it too much for the HP ESXi, even after I tuned it using "idle PC" and modified the Linux hires clock back to 100hz instead of the default 1000hz, it was still eating up over 4000mhz of the 6400hz the system has to offer, and this is just with two routers being simulated . I need to look into what is the least CPU intensive routers to emulate that have all the routing protocols.

QMail toaster is my email server and that is working great amazingly lightweight for the amount of email I have, it uses 250hz and 114MB of ram running on centos 4.
Other things that I'm thinking about creating guests for is an Oracle database probably oracle 9 or something just a toy, don't need it often but every so often I think about playing with Oracle, yes I'm a geek!

Still thinking about adding more memory to the server, 3GB was a good place to start, but I would like to upgrade to 5GB, maybe more if I can find a source of cheap memory for it, I hear they are being carted off to resellers by the truck load so it shouldn't be too hard perhaps one of my readers will give me a great source for the memory for it.

GNS3 rocks

2010-05-12T19:50:00.001-05:00

If you are a Cisco guy you should probably know about GNS3 and Dynamix, they allow you to emulate Cisco switches and routers in your Linux or Windows box, rumor has it, that a OpenSolaris port is in process. It truly is awesome to have a Cisco 7200 class router on your desktop to play with, you shouldn’t use it as a production router but it’s a good way to get ready for your CCNA or CCIE tests and get experience with Cisco routers.

One of the reasons I got the ESXi box was so that I can play with it, without adding yet more ram hogs to my desktop system, just a note, GNS/Dynamix don’t play well with 64bit Vista. It’s much nicer on Linux.

Ubuntu has GNS3 package available just install it. And add the IOS images of your choice. Now one issue I did come across with it is that you have to enable promiscuous mode on the virtual switch or you get a really crazy results, the traffic from your router makes it to the outside world, but your router can’t see any traffic, it looks like a firewall so you get to try and found the firewall.

Note you need to have promiscuous mode allowed in the vswitch in ESX(i) for GNS3/Dynamix to work.

You need to run “IDLE PC” test to limit the CPU usage on the system, as well as install VMware tools to get the benefits of ESX’s memory management.

Top 10 things I learned about using a DL380 G3 as a ESXi LAB

2010-05-12T10:25:00.001-05:00

#10 it’s LOUD – no I’m not one of these overly noise sensitive guys that want mod their PC’s for zero sound. I have 2x PC’s in my living room, both with stock fans, and I don’t notice the noise of them, I have a Cisco switch with a fan that is dyeing that really doesn’t bother me, I have heard 15k rpm SCSI drives with bad bearings, This thing is worse. It blasts the Fans at full speed 24/7. Even with the latest firmware patches and HP drivers installed there seems to be no way in making the thing slow down the fans. Remember and think about this before you buy one for your home, better to have a basement or a room far away from the rest of your daily life for it.

#9 it’s a great server so far, they are selling for a decent price on eBay, I got a dual 3.2ghz processor, dual power supply, 3GB of ram, two drive sleds for $122 shipped. No hard disks included but I have more than a couple SCSI disks collecting dust around the house.

#8 Yes its only 32bit, but for my uses this hasn’t seemed to be an issue, I’m not going to run Solaris/OpenSolaris on it. ZFS loves ram and 64bit address space so its Windows Server and Linux mostly.

#7 have a monitor around, yes it has an iLo (integrated lights out management) but unless you get one with the advanced license activated, you will be limited to boot up messages and static text screens, even a typical Linux/VMware text installer will not show up in the ILO because it detects the fast updating text as graphics.

#6 HP Insight manager is a pig for home usage, it wants 2.5GB of ram to install smoothly, you can leave it at that and tell ESXi to limit it to a much lower amount once you get it installed, but it’s still a pig.

#5 Cheap SCSI Disks are small. 18.2GB used to seem like a lot, but when you start giving 8GB per guest it adds up quickly.

#4 ESXi and OpenSolaris ZFS rock together, I’m storing my guests on my OpenSolaris server with 4x 500 GB sata drives in a raidz pool, not a speed demon but fast enough to keep ESXi happy once I added a small 1.5 GB zVol from another pool ( a single 1.5 TB sata drive) as a slog device really helps. I made it so small so that ZFS's ARC could cache it as much a possible and if the pool isn't busy doing anything else the harddisk head would only have to make small track to track moves giving it the best performance possible so far I have only seen it use less than 300MB max so even the 1.5GB may be too big in this workload. Its nice to see 100MB/s writes to my zpool, and get a gigabit switch if you are going to do this.

#3 the server does a lot, I have had 4 windows 2k3 guests running, and a small linux guest with no performance issues, I’m a SPARC guy I can be pretty patient.

#2 disk and ram are the most important things for ESXi, haven’t really pegged the CPU much during my use so far. Will be adding 2GB of ram bringing the total to 5GB, when the job market picks up I will be replacing the 1GB dim set with 4GB dim set taking the system to 8GB which should be more than enough for my needs.

#1 it’s LOUD – this can’t be over stated, if you live with anyone else that is not tolerant of your toys and the noise they make its probably best to look elsewhere for a ESXi solution.

Bad Disk and destroying the pool, back up in less than 5 hours.

2010-05-04T00:50:00.000-05:00

After about 6 months of being on its death bed my 36GB SCSI hard disk finally died, it had been giving me a few zpool checksum errors from time to time, a few scsi errors in dmesg but it kept working i had cleared scrubbed the pool and cleared the errors 3 or 4 times in the last months, the scsi drive was probably 4-5 years old when I bought it off eBay and I had it another 5 years or so its death was not unexpected. Not sure what I had planned when I put the disk in the raidz pool that holds 2 other 18.2GB drives. Turns out the replacement drive I purchased 6months ago was a few megabytes smaller than the other `18.2GB drives, not sure of the exact difference, it was enough to make zfs that is part of Solaris 10u8 to complain replacement disk too small, this is supposedly fixed in Opensolaris as this is a Sparc box and I’m not ready to deploy sparc opensolaris on my system yet. I needed a solution, So what to do, the pool had 190 filesystems and snapshots cross mounted all over the system and I didn’t feel like scripting some script to copy and recreate the layout. Turns out it wasn’t necessary, 6 commands is all it took to get the system back to normal, and enough disks space to hold the data


zfs snapshot –R pool@now    # creates a snapshot called  @now on every filesystem in the pool. 
zfs send –R pool@now > /largedisk/pool_backup_zfs_stream
zpool destroy pool    # add in –f  if necessary  
zpool create pool raidz c1t0d0 c1t1d0 c2t0d0    # recreate the pool 
cat /largedisk/pool_backup_zfs_stream | zfs recv -Fd -v pool
reboot # restore all the complicated mount points and restart services that failed after the pool destruction

Not only did this restore the pool’s filesystems back to where they left off, it recreated all the snapshots, and mountpoints and all the zfs settings. It did it so well that all programs stored on the pool came up without any issues. I was simply amazed how easy it was to recover the system, including 190 filesystems and snapshots composing nearly 30GB of data in about 3 hours. I don’t think a standard raid5 lvm can be created and mounted in this amount of time much less restoring a complex filesystem structures and all their data.

Tieing a stick to a Dell 2716

2010-04-15T12:33:00.005-05:00

As I talked about in my posting C1721 on a stick, I have acquired a Cisco 1721 router with only 1 fast Ethernet port currently, and I have configured it to be a router on a stick. Here is how to attach the router to a Dell 2716 gigabit switch.

In this example:
port 1 router(192.168.1.101, 192.168.90.101) tags its traffic and also sends other traffic to vlans

Ports 4 and 12 are in vlan 9, but they are not configured to tag or accept vlan tagging.

Step 1:
Login to the Dell 2716, create the vlan

Set the router port to “T” so that it tags packets that flow to the router from this vlan

Then set all other members of the vlan to be U, so that the vlan tags are removed as they are sent to the device, unless your devices in the vlan are configured for vlan tagging.

Now in the vlan port settings, change the non router ports to be pvid of the new lan this makes the switch tag the traffic on the ports, and keeps non vlan traffic off those ports. And disable ingress filtertering so that the ports accept packets that are not tagged with the right pvid/vlan.

This is all you need to go, now you can test your vlan
To test vlan testing you need to add a rule on the hosts outside of the vlan, to point at the router unless you have changed your default gateway.
In my case the routers vlan 1 ip is 192.168.1.101, and the vlan 9 ip’s are 192.168.90.0/24, to make a Solaris host use the new router to route traffic to the vlan you can do


route add 192.168.90.0 netmask 255.255.255.0 192.168.1.101

C1721 on a stick

2010-04-15T11:51:00.001-05:00

Found a good deal on a Cisco c1721 router, and until the extra fast Ethernet port(s) arrive I’m stuck playing with as a router on a stick, here is my simple config.

interface FastEthernet0
description fast ethernet 0
ip address 192.168.1.101 255.255.255.0
speed auto
!
interface FastEthernet0.9
encapsulation dot1Q 9
ip address 192.168.90.101 255.255.255.0
!

Configured the Fastethernet port 0 which is native vlan 1, to have an address.

The second interface fastethernet 0.9 encapsulate dot1q 9, so it tags all traffic that goes into the interface, and I give it an ip address as well though I’m not sure its necessary, but being able to ping it is easy for troubleshooting.

There you have it, I will be adding a blog entry on how to configure a Dell2716 gigabit swtich to be vlan-ed.

OpenSource is running for the Door

2010-04-13T14:20:00.005-05:00

OpenSource is running for the Door at Sun/Oracle. It all began quietly, Bloggers were moving there blogs from blogs.sun.com to private and 3^rd party blog sites, people quietly leaving Sun, giving their fair well notices as their last blog entries. I thought sure it was normal to have some people move on to greener pastures not ready for the change. But as you add names of those leaving its quite a long list, some very big names in the open source community have left, check the google tea leaves.

Searches:

Sun oracle leaving site:blogs.sun.com
Results 1 - 10 of about 216 from blogs.sun.com for leaving sun oracle. (0.22 seconds)
http://www.google.com/search?hl=en&q=leaving+sun+oracle++site%3Ablogs.sun.com

Google has 630 hits on leaving sun oracle site:java.net
http://www.google.com/search?hl=en&q=leaving+sun+oracle++site%3Ajava.net

The trends
http://www.google.com/trends?q=leaving+sun+

Even though some of the search results may be duplicates I’m sure countless other blog entries were not in that list. Then also the big names in Sun’s open source community are leaving, Gosling, Phipps, Marten Mickos, and many others, and for the most part the departures have been quiet, perhaps in the months and years to come we will find the real reasons.
On the other hand what also speaks volumes is the lack of communication about OpenSolaris by Oracle and especially what is holding up the release of the next version, which is currently a hot topic on OGB-discuss mailing list. Even OGB current and former members as well as regular community members are beginning to call for some answers and feed back as to what is happening with OpenSolaris.

[ogb-discuss] Call for Action Ben Rockwood
[ogb-discuss] Call for Action Rich Teer
[ogb-discuss] Call for Action Moinak Ghosh
[ogb-discuss] Call for Action Damian Wojsław
[ogb-discuss] Call for Action Joerg Schilling
[ogb-discuss] Call for Action Damian Wojsław
[ogb-discuss] Call for Action Dennis Clarke
[ogb-discuss] Call for Action Al Hopper
[ogb-discuss] Call for Action Octave Orgeron

Up to now I thought that Oracle was going to do a lot for open source that Sun came along with the purchase, because no matter what else was going on Oracle likes acquiring companies and many of the companies it has chose to purchase were open source based or had strong links to open source and given Oracle’s pains of getting the European Government buy in on the Sun purchase it would do a lot more to keep open source community happy. But from the looks of it, the next purchase that Oracle makes may not go anywhere near as smooth as Sun if the people and the Open Solaris community members as witnesses, if Oracle doesn't start being more communicative.

One more ZFS video

2010-03-09T14:32:00.001-06:00

Dynamic LUN expansion

http://blogs.sun.com/video/entry/zfs_dynamic_lun_expansion

Cool ZFS Dedup video

2010-03-09T14:30:00.001-06:00

Check out this vide by George Wilson of the ZFS team.

http://blogs.sun.com/video/entry/zfs_dedup

VDI Sun -- Part 4a the code.

2010-02-24T14:03:00.002-06:00

Here is the latest version, it compiles and runs without crashing if you have root or the right priv's so feel free to use it and look at the code and laugh...

it uses kstats so it doesn't just allocate memory blindly.

smart_lockngo.c

VDI Sun -- part 4 battling locked pages.

2010-02-22T20:23:00.002-06:00

Okay I’m learning a lot about memory allocation more specifically how and when pages are locked into physical memory and which kernel functions are responsible for it. While trying to figure out why pageslocked kept growing in kstats and what is using it. I first blamed it on my C program being too aggressive and just locking memory blindly now that it’s being started by cron without human intervention, so I am currently working on it and giving it some intelligence of about locked memory, but still even with my improvements pageslocked kept growing so. I brought out the big guns, DTrace which I tried to catch VirtualBox locking memory, and trying to see how much memory is being locked in the VirtualBox tasks using pmap –x and not really finding a good answer, my current guess is that Virtualbox’s kernel modules are locking memory using kernel functions, and this really isn’t triggered by a normal syscall. After going away and finding pagelocked still growing even though there are no VirtualBox guests running on the box at all, it finally clicked in my mind ZFS ARC is using more and more locked pages, upto max arc which I set to 4GB or about 66% of free memory on my system. Now ZFS will give it back, but not fast enough if you ask for it a gigabyte at a time and immediately lock it into physical memory.

While fighting pagelocked growing was bad enough, the reason I’m working this hard is that I am running into this bug http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6910378, while the warning messages are bad enough, it’s the underlying upheaval that it creates. Network connections die, iscsi initiators on remote hosts timeout, which taught me how to do forcibly remove iSCSI luns that are formatted with XFS, can’t login after a few minutes the box gets back to normal.

To add more pain, I find the bug and see that it’s fixed in b133 and was released into OSOL /dev on Friday night. So since I am currently tracking /dev, for ZFS dedup, I figured I might as well upgrade. Well the upgrade seemed to go well, packages are downloaded, new versions are installed, cleanup scripts all run, and I reboot the box. Now since I rarely use the gui on the system, I don’t have it connected to a monitor or even a keyboard. So I go into shell on another box and run ping –s waiting for the system to come back to life so I can login. 5 minutes go by, then 10 min and finally after 15 minutes of the system sitting there, with no disk activity lights, I figure it’s time to connect a monitor and keyboard, which requires a restart because video is not enabled on the machine unless a monitor is connected on startup. So I reboot, tell it to grub to boot into text mode, add –v for verbose output, remove the graphic screen and not apply special colors to the screen all of which makes kernel output unreadable until X starts. So finally I see Solaris dumping out tons of debugging information, all seems pretty normal for a –v enabled Solaris boot, after a while I see my iSCSI connections failing, but I still can’t ping the box, so I think it is the iSCSI SMF manifest screwed up and starting before physical networking is started. Finally a login prompt appears and I’m able to get into the console, I run ifconfig –a, and see all my normal network interfaces up, lo, Virtualbox’s virtual NIC is there, but where is my gigabit adapter, that is strange. I look in dmesg, and nope no mention of my NIC, run scanpci and it doesn’t see my NIC. So I decide to try and plumb it, but no luck it’s not finding my NIC, well since OSOL has the best of live upgrade enabled using ZFS, I simply tell beadm to activate my earlier version, and reboot the box.

ifconfig rge0 plumb

beadm activate opensolaris-dedup-7

Shutdown –i6 –y –g0 ; sleep 300; reboot

Not sure why shutdown doesn’t always reboot the box so I forcibly reboot it after giving it a chance to update the boot archives. About 6minutes later the system is back up and running, network is fully intact, and everything is good to go, hopefully they will fix the but.

Since it was ZFS’s arc that is locking memory I have gone back to setting arc maximum to 3GB I had set it to 4GB and that is fine for a while but then starts to grow slowly and consumes 66% of memory leaving little memory left for VirtualBox guests, and I’m going to try and keep pageslocked at 75% of physical memory, that will leave about 3GB of ZFS ARC + 1.5GB of virtuals, with small Windows XP instances needing between 384 and 512MB that is more than a few desktops that can be loaded. May even drop Max ARC to 2GB, once I deal with memory pressure issues it should be fine, but this is a work in progress.

Hopefully tomorrow I will post an updated version of my code.

Update: finally posted the code here smart_lockngo.c

VDI part 3 and DEDUP can lead to system instability.

2010-02-18T12:13:00.002-06:00

I hope you all aren’t bored with VDI yet, but at least I’m posting again. I do find VDI incredibly cool and powerful technology. I can clone Virtual Box guests in less than 10 seconds once the guest image is imported, then I can try something and throw it away when I’m done.

Now onto some growing pains and work a rounds for them. I followed the VDI demo config which says you need to limit ZFS ARC to 2GB, which is all good and fine if you are only running VDI on the box, but of course I’m not. I am running OSOL b132, with dedup enabled(at least on some file systems) on the VirtualBox host, and if you follow zfs-discuss you will notice a disturbing number of occurrence of ZFS taking long time to do things on file systems that have dedup enabled, not only are writes slow, unless you have a slog (separate logging device(s)) and large amounts of ram, but zfs destroy can take long times and if not allowed to complete, on reboot zpool importing can take even longer, some reports up to 60 hours without use of the machine.

Last night I decided to do some clean up on my system, I thought I had cleaned out the file system, using rm –rf that really doesn’t impose the large penalty of a zfs destroy at least on b132, and I thought I was good. But the –f means don’t bug me if I do fail so I ended up with 12GB of files left on the file system when I ran zfs destroy since the system didn’t return immediately, I dug into my ZFS tool kit of utilities, and tried zilstat which showed little activity, then I moved onto arcstat which showed about 30% arc cache misses, and iopattern (that is part of Brendan Gregg’s DTrace toolkit) showed 100% random IO thus why it was taking so long, this proceeded for over an hour, the box was still quite usable, and responsive then arcstat began improving I was only having less than 10% cache misses which lead to my hope that it would be done soon, but then the box decided not to respond, it had decent ping times, but ssh connections were timing out. I happen to have gkrellm running over ssh –X which was periodically updating and I saw one core pegged at 100%, the other less than 5% busy, and the disk was doing less than 500kB/s which is bad for a 1TB SATA drive in a zpool by itself usually capable of delivering 80MB/s or more. The box did eventually get back to normal. In the end Linux box that was using an iSCSI target on the pool timed out and was pretty unhappy and had to be rebooted, but that was the worst of it.

Since that happened I decided to change the ARC max limit I set in /etc/system, it is now set to 4GB max ARC, and I have put lockNgo the program I mentioned in part 2 of this series into root’s crontab and now executes every 5 minutes, when I woke up this morning I logged into a VDI desktop, and all went smoothly and there was 1.5GB of free ram available on the system. I may try removing the arc limitation and just adjusting the amount of memory that lockNgo forces to be available on the system, currently though I do give up the use of 1GB of ram, it’s better than limiting ARC to 2GB on a fileserver. I should also be able to tune the cronjob not to run 24/7. I’m not going to need a virtual at 3am so why reserve the memory. I still haven’t got around to doing any more zfs destroy’s but I will have to do more clean up eventually.

My VDI setup currently is using 2 boxes, the Sun Blade1000 is running Solaris 10u8, VDI core, clustered Mysql, and OpenDS, as well as acting as the iSCSI targets for virtuals so that it doesn’t compete with VirtualBox on the AMD machine for memory, and will soon be expanding to include Squid and DNS so I can power down my Sun Blade 1500 (Frankenstein), to save a bit of power since the blade1000 isn’t really taxed running VDI. I am currently serving Virtual Desktops on upto 3 Sun Ray 1’s and possibly on Sun Desktop Client which is a virtual Sun Ray 1 that runs on windows. I haven’t started up more than 2 Virtual desktop yet but have more testing to do.

Lock N go or part 2 of VDI for home.

2010-02-16T13:13:00.002-06:00

As mentioned in VDI for home blog entry I needed to write a small C program that locks some memory and then free it forcing Solaris to push out cached data, and memory being used by executable s. So here it is, it’s been tested and works for me, your mileage may vary, its currently limited to only one command line argument, I’m sure it’s just a few more lines to convert it to many command line variables but I figure I needed to get this out in the world before I forget about it.
It’s based on James Litchfield's blog entry on locking memory found at Locking Memory

Download the code at lockNgo.c

To build, use either cc or gcc

gcc –o lockNgo lockNgo.c or cc –o lockNgo lockNgo.c

To run you need to be root or granted PRIV_PROC_LOCK_MEMORY privilege as documented in James’s blog above, pass –h to see arguments that can be used.

NOTE: I am not a professional C coder, nor do I wish to be one, fixes and modifications gladly accepted.

A reason to move to the Netherland

2010-02-12T13:08:00.002-06:00

200mbit internet, Netherlands Town, need i say more?

VDI for the Home

2010-02-12T11:05:00.003-06:00

I have been hearing a lot about Sun’s, oops Oracle’s virtual desktop solution, VDI and decided to give it a try. Of course being a home user on a budget these days, but yet with an enough old Sun gear to choke a horse I figured it wouldn’t be an issue. I figured I would be able to coble something together. I’m sure it’s not an issue for a typical business user to come up with one or more dual or quad core system with 8+ GB of ram so they can install Solaris 10 update 8 for this project, of course the only box at home I have that comes close to meeting the specs of a demo machine (x86 four or more cores and 8GB of ram, running Solaris 10u7 and preferably two hard disks) is my Fileserver, that is currently running OSOL /dev so I can play with ZFS dedup, there is no way to put Solaris 10 on it because I would lose access to all my data. It’s a fairly beefy box for home use a 2.4 GHz Dual core AMD Athlon with 1MB L2 per core, 6GB of ram. 4x SATA 500GB drives, a separate 1TB drive for a playground pool, and 250 GB SATA drive for rpool and a gigabit Ethernet.

Let the creative solution and rule breaking begin. I decided to put the core VDA server inside a virtual running Solaris 10 inside VirtualBox, which is not a supported configuration, and allocate it 1.5 GB of ram, which goes pretty well, I followed the directions and modified the /etc/system file in the main box, so that ZFS’s ARC wouldn’t use more than 2GB of ram. It installs fairly painlessly, the next step was to install VirtualBox which I thought I could ignore since on the raw hardware I have the latest VirtualBox version 3.1.12 installed, but of course, this isn’t good enough for VDI, as it has its own custom VirtualBox release with GPL’d extensions to the webserver interface. So I decide to bite the bullet and install there custom version using their script, well to make a long story short, after getting a OpenSolaris 3.0.8 version of the custom VirtualBox from the web and then hacking the scripts, to install the webserver extensions onto OpenSolaris and reconfiguring my currently running apache to deal with VirtualBox webserver interface. The next headache appeared, I had upgraded my OSOL box to use Comstar for iSCSI, which is incompatible with VDI, and currently there is no complaint by iSCSI when you try to use it to manage iSCSI targets even though Comstar is running and has no intentions of serving those targets. So I had to remove Comstar which requires booting to another boot environment, and removing Comstar and then rebooting to the current boot environment. Of course I will have to reverse this change because I hear Comstar will be supported in a future release as normal iSCSI is going away. The next step was to start importing a Desktop to use as a template; you basically just install the desktop of your choice in a Virtual, and then install VirtualBox guest tools, and any other software you can’t live without, Firefox, and the extensions, etc then shutdown the Virtual. Go into the VDI web interface and import it as a desktop, which on my machine took about an hour.

While the import is going on it’s a perfect time to deal with a Directory Service. Here is a good short tutorial how to automate the process of installing OpenDS. It’s all very smooth, and only requires you to enter one password, unlike other directory servers I have tried. Once you install it, go into control-panel app, that it starts for you at the end of the process and Directory Data, and then Manage Entries, Domain, People and edit a couple of the entries it created for you and set to your usernames and set a new password. By the way not sure I did something weird, but my copy of OpenDS ended up getting installed at /OpenDS so the path to the control-panel app is /OpenDS/bin/ which made it difficult to find the next day as it was in the last place a good sys-admin would look for it.

Now you can go back into VDI web application and set your LDAP server, the anonymous settings is good enough for OpenDS, here is a major tip on how to make it work, this is the entry you need to use for it to find your entries, its ou= not the example. This had me banging my head against the wall. It all appears to look like all of VDI should work but, the remote desktop program (uttsc) that comes with it fails to work, rdesktop works, windows remote desktop works, but not Sun’s.

Important Tip: for OpenDS users here is main important line to the LDAP configuration

ou=People,dc=yourdomain,dc=com

You can go to the users section of the VDI web interface and search for your user name that you modified while configuring OpenDS.

After the desktop import is complete, you can convert it to a template. Then clone the template as a desktop, this is where you can see the magic of ZFS in action, the cloning takes less than 15seconds on my box to deploy a working XP instance isn’t bad in my book.

I have VDI working for the most part, the next issue, VDI monitors the VirtualBox systems ram by looking at available ram, now Solaris applications regularly access disk and the data is cached, so this available memory number is really low, usually 200MB or so. And VDI refuses to start a new virtual on the machine because available ram is too low. The Solution to this is to use VirtualBox GUI to start up a virtual allocate it to allocate 2GB of ram or so, which forces the OS to move data to swap and flush the cache, then exit the Virtual which is mlocked (locked into physical memory) and then power-off the virtual, then VDI is happy and will start up virtuals. The long term solution is to write a small C program that mlock’s gigabyte of memory, and then exits and put it in a cron job.

Well it turns out the memory availability issue was too painful, so I decided to move the core VDA server out of a virtual and to raw hardware, as the guides recommend 2GB of ram, I chose my Blade 1000, for this task it has 2x 750mhz CPU’s and 4GB of ram and 2x 146GB FCAL hard drives in a mirrored zpool, I also took advantage of the dual gigabit NIC card on board. This gave me a bit more room for memory usage on the OpenSolaris fileserver, but it seems to be a losing battle until I write the C program I mentioned above to free memory for Virtuals. I also moved the storage for the virtual hosts to my Blade 1000, as it has plenty of fast storage for the task, note: iSCSI needs a gigabit network, the synchronous method used for iSCSI pretty much demands it at least for application, I tried with the on board 100mbit NIC and it limited through put to 4MB/s far below 100mbit’s maximum of 12.5MB/s, with the gigabit NIC the through put jumped to 6MB/s because of the lower latency, which gave me about a 12% drop in performance over local storage. The situation is a bit better now, but I’m going to write the program to keep memory free. But all in all VDI is really nice software, and I have only started to scratch the service as to what it can do. I am still looking at what else I could do to improve performance using existing machines that are currently powered on.

Related Links:

Pctures of VDI at JavaOne

Sun VDI Forum

How to increase Logging level

Oracle makes commitments to MYSQL users

2009-12-14T10:31:00.003-06:00

In case any MySQL supporters missed this, Oracle made some commitments that should hopefully make he UK anti trust committees happy and reassure MySQL users as well

Oracle Makes Commitments to MySQL users and 3rd party apps

DeDup... WOW

2009-12-14T10:27:00.002-06:00

Okay we all have heard about how cool dedup is production environments like for backups, and vmware or other virtual environments where you have lots of repeated data, but I am simply amazed at how much duplicate data I have on my home boxes, so far just copying my directories from the old non-dedupped pool to the one that I have converted to the latest zpool version and enabled dedup is amazing, I haven’t even finished copying everything yet, but I have done 592GB so far and have a dedup ratio of 1.43 which gives me about 254GB of duplicate data and that is equal to free disk space, this is all without trying to produce duplicated blocks. Now another amazing fact is that most of this data was pdf’s, mp3’s and video files, and iso’s for OSes all of these file types mostly contain compressed data or were compressed using bzip2 or gzip, yet ZFS was able to squeeze the data even space saving out of disk.

Since I frequent #zfs on freenode.org, I guess I hear more users coming online asking about what is the best motherboard, controller, sata drive layout for their killer home fileserver, I have seen users talk about having a dozen or more 1.5TB drives, in a raidz2 layout so approximately 10TB of usable space. It always amazes me how a home user can use that much space, I have 1.5TB of storage in my raidz pool, which is huge and took quite a while to fill up with stuff that mostly I won’t ever use, but can’t stand to part with because I might need, and there is no real way for a home user to back up large quantites of data, DVD’s aren’t really an option, I don’t know of an application that burns dvd’s and offers an interface that will sit and ask you to insert your next DVD… 20 years ago there was a program that did this with floppies but I may be missing something but this really isn’t available for Solaris yet, not sure I would sit there and feed 100 DVD-r’s into my fileserver, and it would probably take a week to backup 25% of my small 1.5 TB fileserver, bluray burners are just too expensive as is the media, and 1TB over DSL or cable is just too painful to think about and costly as well.

What I have found is the more space someone has the less likely they are to delete stuff that they should. There is also a much greater chance of having files just plain missplaced, while moving files to the dedupped pool I noticed, files that I had moved from other older boxes to my large raidz pool that were hidden away in [machinename]/home/[username]/.local/share/Trash … oops those were deleted month or years ago and were supposed to be gone but some applet hid them away and the applet will never delete them so there they sit wasting disk space.

I can only imagine what people who have these monster ZFS fileservers with 10+ TB of disk space will find one day… I guess it will really be fun for those geeks that want to try and fill there messavie mega ZFS servers and dedup keeps finding all the duplicates so there attempts at filling them run into a brick wall after about 5TB or so because they won’t be able to find enough unique data they are interested in, come on how much anime, and rock mp3’s are there out there once you dedup them ;-)

Tracking dedup

2009-12-10T15:32:00.002-06:00

If you haven’t been living under a rock for the last month, you know that ZFS got deduplication at least in OpenSolaris, so I decided to give it a try, it all works in typical ZFS simplicity as documented at Ben Rockwood’s blog You can see that currently ZFS gives very little hint that anything is happen differently than before, there is dedupratio, but that really doesn’t give you any realtime information about the process.


amd:~$ zpool get dedupratio  puddle
NAME    PROPERTY    VALUE  SOURCE
puddle  dedupratio  1.17x  -

So what is there to do? Since this is based on Solaris 10 and beyond, DTrace comes to the rescue, after a while of googling and searching in the OpenSolaris source code browser, and finding the change summary for dedup at http://mail.opensolaris.org/pipermail/onnv-notify/2009-November/010683.html I found the key, ddt.c, it’s the file where most of dedup is hidden, comments are of course very sparse in the file. So I had to guess but I think I found the function ddt_object_update that updates when a duplicate is found, I know, this only scratches the surface of dedup, but I wanted to see it working in real time. I’m sure some ZFS guru will chime in here and tell me that I got the wrong function, but from my testing, I think I got the correct function, I ‘m not even going to try and decode the pointers to pointers used in the arguments that are passed. But at least it works for me.


bash-4.0# dtrace -qn 'fbt:zfs:ddt_object_update:entry {c++;t++} tick-1s {printf("Duplicate blocks %ld found this second for a total of %ld duplicate blocks since script started\n", c, t); c=0;} '
Duplicate blocks 0 found this second for a total of 0 duplicate blocks since script started
Duplicate blocks 0 found this second for a total of 0 duplicate blocks since script started
Duplicate blocks 0 found this second for a total of 0 duplicate blocks since script started
Duplicate blocks 0 found this second for a total of 0 duplicate blocks since script started
Duplicate blocks 79 found this second for a total of 79 duplicate blocks since script started

Note: this really isn’t real time, since ZFS is a transactional database, simple writes can take over 30 seconds for them to be flushed and ZFS takes a bit longer to do the duplicate block accounting, at least that is what I determined in my testing.

1800 miles, 4 states...

2009-11-11T15:49:00.002-06:00

1800 miles, 4 states, 2 hours on the tarmac, 1 hour job interview, 20 hour day, now I’m home.

Yesterday started at 4:30am with my alarm going off, after a fitful 3 hours of sleep, yes I’m a procrastinator was up to midnight packing the laptop, making sure the phone and the mp3 player was fully charged and had new stuff to listen too. Throw on clothes head off to the airport. And get on the plane to Memphis, Tennessee at 6am, as it seems with all planes I have been on lately they have been full, well I thought I was going to get lucky and have an empty seat next me, or I did until about 3 minutes before the plane’s door was closed and then the seat was at least filled, but all and all a pretty non-eventful flight.

In the intervening hour, mostly spent going between concourses and hitting the Starbucks standing in line, getting a Latte and a piece of pumpkin cake and then heading off to the next plane getting there in a few minutes before the 30 minute pre-flight boarding, flight was nice, but full but quite nice and smooth despite the remnants of hurricane Ida making Charlotte, NC very rainy and wet.

Landed met with the Agency person, and had a nice lunch of Red Robin, then off to the interview still about 30 minutes early after going through 2 security stations, and got to sit in the lobby for 30 minutes.

Interview went well I thought, they ran out questions to ask because this was now the 3^rd time we had talked, it was only an hour, did I really sign up to fly 1800 miles and a 20 hour day for a 1 hour interview?, at least the agency paid for the flight. Went back to the Airport was about 4 hours early finally tracked down the right concourse and gate, my flight wasn’t even on the departure screens yet it was that early, had a decent Chinese food at a fast food place in the Airport, then proceeded to the gate and found an outlet so I could recharge my laptop and mp3 player that managed to power itself on and play during lunch and the interview using half its power.

About half an hour later still 2 hours before the flight they called all the passengers waiting for the 7:15 flight up to the gate, and told us our flight was delayed because of the weather in Atlanta, and wanted to move us to the earlier flight, which was really the 5:35pm flight that was now scheduled to leave at 6:40. Well I thought okay I don’t care since I can sit in any airport, I already had an hour layover in Atlanta anyway.

So after boarding the 6:40 flight which seemed to take much longer than usual because they had to put everyone from the earlier flight and the later flight onto the plane we leave the gate, taxied to the designated waiting zone, to sit on the tarmac, for about 30 minutes, then they turn off the no electronic device sign so we can listen to our mp3’s and laptop and even cell phone because we have to wait her 15minutes more before we can get into the official take off line, then 10 minutes later they turned the signs back on because we were going to get in line for takeoff but it was just tease because after the engine test the turned the signs back off because for some reason it would be an additional 30 minutes, well to make a long story short we actually got to the takeoff line at 7:55 pm, which was 40 minutes after the originally scheduled take off time of the later flight and 2 hours and 45 minutes after we boarded the plane.

To add insult to injury when we got into the air they announced that because the flight was so short about 1 hour and 2 minutes there would be no beverage service for coach how nice, strange I have had drink service during a 30 minute flight between Milwaukee and Chicago, we were held hostage in the plane for 3 and ½ hours and they couldn’t even be bothered to bring out the drink cart, at least they did offer cups of water to those that wanted it on the flight. I think it should be a rule that the time stuck on the tarmac should be included in the flight time calculations and they should be forced to provide beverages and snacks accordingly.

Well make it to Atlanta finally the gate staff which should have been apologetic seemed to care less that the 5:32 pm flight arrived 4 hours late at 9:40pm, and could be barely bothered to check the status of my flight to Milwaukee that was supposed to leave 9:42pm, which luckily delayed to 10:20pm so I got to run through the airport to reach my flight which of course was on a different concourse from where we landed.

As hard as it is to believe the I actually got an empty seat next to me on the final leg of the flight into Milwaukee and full drink service even thought the flight was only scheduled to last 28 minutes longer than the haha 1 hour and 2 minute flight from Charlotte to Atlanta. At 11:30pm I finally arrived back in Milwaukee 18 hours and 30 minutes after I left on my journey,

Home at last, now I just have to wait to see if I get the job.

Got dual cores/cpu or more? or maybe just more than one disk.

2009-10-13T18:22:00.003-05:00

I came across Thomas Maier-Komor’s home page when I found this cool utility he wrote xjobs, it takes what xargs does and makes it multicore aware so that it starts x number of jobs to get things done faster, which is really cool to me. Before xjobs I found my self always writing little scripts like below, so I can use all my cores and may even take advantage data that is cached and have the operation complete faster.
for I in *.zip ; do unzip $i & done
But this doesn’t allow me to control how many processes get started off at any given time. This is where xjobs comes in. It basically does the same thing but starts the number of processes as the system has processors, but of course this is configurable. Since I didn’t want to repeatedly uncompress files to test this, I decided to throw the fact that my systems have multiple disks and file systems and look at how xjobs can help with IO spreading as well as just the CPU load.
The task that I am testing with is find all copies of “tar” I have in /usr and /opt, traditionally I just use find, but that really isn’t very efficient since it looks at one file system at a time. I ran all of these commands multiple times, to be sure the Solaris has cached all relative metadata so none of the commands should require physical IO to the disks. First on my Dual core AMD box, /opt is on my non-root pool composed of 4x 512GB sata disks, and /usr is on my root pool made of a single 250GB sata disk.
Xjobs arguments used –j sets the number of jobs to use to complete the task and –v 0 tells it to be quient about process starting and ending.
To make find, xjobs friendly I wrote a small wrapper script

qfind:
#!/usr/bin/bash /usr/bin/find $2 -name $1 2> /dev/null

The actual tests:
#xjobs using two processes. $ time echo -e "/usr \\n/opt \n" | xjobs -j 2 -v 0 ~jamesd/xjobs/qfind tar /opt/csw/bin/tar /usr/bin/tar /usr/gnu/bin/tar /usr/sbin/tar real 0m2.159s user 0m0.389s sys 0m2.838s #xjobs using just one process $ time echo -e "/usr \\n/opt \n" | xjobs -j 1 -v 0 ~jamesd/xjobs/qfind tar /usr/bin/tar /usr/gnu/bin/tar /usr/sbin/tar /opt/csw/bin/tar real 0m2.825s user 0m0.375s sys 0m2.443s #the old fashion way, using find $ time /usr/bin/find /usr /opt -name 'tar' 2> /dev/null /usr/bin/tar /usr/gnu/bin/tar /usr/sbin/tar /opt/csw/bin/tar real 0m3.006s user 0m0.365s sys 0m2.633s

Now what surprised me the most, is that old fashioned find was slower than even xjobs when I limited it to just one process. Even with the extra work to spawn the xjob with one thread is a lot greater because it has to echo the two directories into xjobs and has to setup a pipe as well, and then run a script two times that spawns two copies of bash where find is just one command doing what it was designed to do.

Okay now if you have only one cpu but multiple drives does his help you? Let’s see. For this test I use my Blade 1500, single cpu, and 1 ide drive formatted with UFS for /usr and /opt on a zpool raidz composed of 3x 18GB 10k rpm drives.


frankenstein:xjobs-20091012$ time /usr/bin/find /usr /opt -name 'tar' 2> /dev/null\                /usr/bin/tar
/usr/sbin/tar
/usr/sfw/share/doc/ant/manual/api/org/apache/tools/tar
/opt/csw/bin/tar

real    0m5.827s
user    0m0.729s
sys     0m3.823s

frankenstein:xjobs-20091012$ time echo -e "/usr\n/opt"  | /usr/local/bin/xjobs -v 0 -j 1  ./qfind tar
/usr/bin/tar
/usr/sbin/tar
/usr/sfw/share/doc/ant/manual/api/org/apache/tools/tar
/opt/csw/bin/tar

real    0m5.969s
user    0m0.736s
sys     0m4.034s

frankenstein:xjobs-20091012$ time echo -e "/usr\n/opt"  | /usr/local/bin/xjobs -v 0 -j 2  ./qfind tar
/opt/csw/bin/tar
/usr/bin/tar
/usr/sbin/tar
/usr/sfw/share/doc/ant/manual/api/org/apache/tools/tar

real    0m5.264s
user    0m0.726s
sys     0m3.604s

The results show that xjobs was about .6 second or about a 10% gain by using xjobs, Since we didn’t have to access physical disks for any of these tests I think we can assume the results would be more impressive if the cache was cold.

Check out Thomas’s home page he has some other cool utilities, like multi-threaded tar patch and sysstat.

ZFS ARC summary in web language

2009-09-16T17:05:00.006-05:00

Lately I have been working on system monitoring scripts and decided to take Ben Rockwood’s ZFS ARC summary script and convert it to output in html format. This is just the first step I will be working on a configuration file to use MRTG to track this information s well. You can get the current release of this script at zfs_status_web.txt

Sample output, the script looks nicer because it doesn't conflict with my blog's CSS.

System Memory
Physical RAM	6006 MB
Free Memory	1062 MB
LotsFree	93 MB

ARC Size
Current Size	2115 MB (arcsize)
Target Size (Adaptive)	3298 MB (c)
Min Size (Hard Limit)	622 MB (zfs_arc_min)
Max Size (Hard Limit:	4982 MB (zfs_arc_max)

ARC Size Breakdown
Most Recently Used Cache Size	57% 1910 MB (p)
Most Frequently Used Cache Size	42% 1387 MB (c-p)

Tracking down sunray's on ciscos

2009-09-11T04:29:00.005-05:00

After installing mrtg/rrdtool on OpenSolaris Tracking down I wanted to see how much bandwidth my Sun Rays were using so I needed to know which port the sunray were plugged into on my Ciscos

Normally the last 2 lines is enough but ping is necessary at least on my Cisco 2950's and sunray's


ping [ipaddress]
show arp | include [ipaddress]
show mac-address-table address [mac address]

based on information I found at: http://www.velocityreviews.com/forums/t37560-how-to-track-down-whos-on-what-port-on-an-ios-6509.html

and for those that are interested here is the bandwidth usage of the sunray, its sitting idle and it uses very little bandwidth.

Happy B-Day to me!

2009-08-17T19:09:00.001-05:00

It's my Birthday, wasn't it just my birthday 365 days ago? sure feels like it. It's going okay, got news that I have a 2nd interview lined up for a job I'm trying for. So seems like a good day so far.

Are you old enough to get the humor in this?

2009-08-04T10:57:00.002-05:00

My friend Melanie posted this really funny skit, on her blog check it out.

Another funny story you don't have to old to get it, just married.

Interesting Links and Tools

2009-08-04T09:28:00.002-05:00

Unix isn’t going away, and 3 that it may.
Found it really interesting that 1/3 of all server sales by dollar are Unix Servers.

Freebase Parallax
Cool search engine that changes the way you search from one topic to one page, a more to a more relationship type search. watch the 10 minute video on the home page.

VirtualBox 3.02 released

One of the best free virtualization tools, new version, supports SMP in virtuals now and many other features, Virtual box keeps getting better and better.

FoxTab
One of the coolest Firefox plugins gives you a 3D w to choose which tab you want, works great when you have browser windows open and each has more than 10 tabs. Gives you a way to show off that fancy 3D video card, with out loading a game.

Sync Back
Need to backup your home windows boxes? Syncback is a great way to do it, backs up to directories, windows shares, even my favorite FTP

WinDirStat
Gives you a way to find all those large that you know they are somewhere, but not sure where, and allows you to see what is really using all your hard disk space.

2 AM coding session almost goes bad

2009-08-03T12:03:00.002-05:00

Well it figures, 2am the most wondrous and dangerous time for coders and sys-administrators alike, it’s when we come up with the best ideas fix bugs we fail to notice at any other time, perhaps even bugs we introduced in 2 AM coding sessions, it’s also when we manage to do the most damage to our projects.

Back story:

I have a home intranet that pulls and displays RSS and Atom feeds as well from the internet, as well as prints lists of Hyperlinks these are all stored in MySQL tables and they are classified by content type, General, Weather, Geek, and Sports. A function generates the SQL query to display what each family member is interested in. When I started it all works beautifully.

Tonight:

I got tired of seeing the RSS feed from WhiteHouse.gov, I think I could delete it, but nah it would be better to add a way to simply disable it for now since I’m burned out on politics at the moment and perhaps re-enable it later when the next election comes around. Sounds easy enough, right? I fire up PhpMyAdmin and make the necessary changes to the tables, I add a field to each of the tables, called “enabled” set the default value to 1 being that there are only 3 tables that are needed and they are rather small tables, less than 300 rows it all goes smoothly.

Now on to changing the PHP code, I fire up NetBeans 6.5.1 on my desktop, that has my Intranet site in a project and go about making the simple change to make the query generation function only display enabled rows. As NetBeans now understands PHP code since somewhere after 6.5. I can make my change and have reasonable confidence that my change will run.

$output .= ‘and `enabled`= 1 and ( ‘ ;

I click the little green triangle (play button) on the toolbar that tells NetBeans to run the project, so first NetBeans uses ftp to upload the changed file up to my web server and starts Firefox to display the intranet. As expected it goes fine. So I commit my changes to the source repository (I use subversion but most of the other ones work as well), yes also handled painlessly from inside NetBeans.

Now here is where I should have gone to bed, Do I? Of course not, I think maybe at some point I will want to display disabled the records I should make it possible as well. The new code to do this looks simple as well, but did I mention that it’s around 2am?

$output .= "and `enabled`= $this->prv_enabled and ( " ;

Because I now have a variable that must be replaced in the string I need to change from ‘(tick) to quotation marks, which I remember to do, then add a small function to change the newly created private variable. I also modified the constructor function to set the new variable. Add comments and fix a few other small things I see along the way. Now time to test, click on run project, NetBeans once again uploads the new version of the file, and tells Firefox to display to my home page, to my dismay, this time Firefox Renders my code to a blank page.

I think it’s not the end of the world. I flip back to NetBeans and take another look at my code. NetBeans shows the code to be good. Still no hint to what has happened. Just before deciding to hit undo all my changes I made one by one, I remember that NetBeans has local reversion history built in, I click on the Versioning menu, and choose local history Local history and up comes a window with a side by side display of the difference in each code change look at the first change that made today and all is well, then I click on the second change still good. I continue through all the changes twice, then I notice a red dash (this mark indicates that there is an error further down in the file.) had appeared on the right hand of the display, further down the window in one of the middle change sets. I then find a change I don’t remember making and hadn’t noticed in my previous reviews, did I mention it was after 2am? So I scroll down to that change, and all the code in a few lines further down were all marked as errors so I knew I had found the bad line, now as it was after 2:30, I was thankful to see a red X by the change, clicking it worked as expected and reverted the change, I didn’t have to figure out what changed, a simple click and it reverted the line by back. And all the bad code notifications went away, since this worked so well. I thought okay let’s see if what the problem was, and went to the edit menu and clicked undo, and the change went back in and I studied the line in question and found that I somehow I ended up changing a ‘ (tick) to the a quotation in this part of the file as well, which is strange since I did the first change by hand, but NetBeans made it relatively easy to find even in my sleep deprived state.

An hour worth of work saved and multiple hours of debugging the issue of a random change averted thanks to NetBeans, I fear how much work it would have been to find this bug in vi.

Happy Fathers Day

2009-06-21T16:18:00.002-05:00

Happy Father day to all the Fathers out there. Have a great day.

serve weather

2009-06-19T01:11:00.002-05:00

its hard to sleep when there is this many bright colors on the weather map, and they are expecting hail.

those in california aren't used to seeing this many colors on the radar map all the colors are usually associated with loud noise from thunder, lightning, and possibly hail, which is expected in the next few minutes.

Roy Wood has a really interesting number

2009-06-18T15:42:00.002-05:00

Just read this really interesting blog entry about an interesting number he recived in an email.

Its really cool. Check it out.

SXCE b115, not ready for prime time

2009-06-10T00:57:00.006-05:00

Decided to upgrade my b86 SXCE box to something more recent. But what I got was lots of broken bits instead. Wanted to play with the latest bits, Cross bow, DTrace extensions, etc.

My AMD box was perfectly stable and well performing on b86, I expected a smooth upgrade as usual, but I ran into a lot of headaches.

After what seemed like a nice smooth install, I booted it for the first time, using xVM mode, and on its way to boot up it crashed. Okay i don't mind a few rough edges, and booted back into normal mode with no xVM, which proceeded as expected.

Next step was to do a zpool import and see if it could find my data zpool named "tank", the system found 3 of the 4 drives that make up tank, okay that is scary since it found 3 of the 4 drives, some of the drives were on the motherboard, it found 1 of them as well as the root drive that is in a seperate pool, and it found 2 drives on a pci sata controller, i reseated the power cable and sata cables and tried again, still no luck. I moved the drive that wasn't comping up to a different port on the mother board, still no luck, finally giving up i moved it to the add on card, which is on pci-32bit and it worked. Okay I can breath a bit easier now at least it seems all my data. About 1TB of data, that is mostly not backed up , how do home users backup 1TB of data? not easily.. you keep backups of the important stuff and keep snapshots.

Well I though okay my data is in tact now to get the machine back to doing its job.

Restore passwd shadow group named.conf /etc/bind files to the place and restart dns okay that was easy thanks to the copies of those files that sit on my home directory on the pool i just imported.

Now to get Sun Ray services back up and running. After installing and following the directions, i found that the SRSS still is not happy with the changes to the sockets made back before b107 Bug ID#6799655 even though the bug is labeled fix it still requires the work around and a reboot. So another hour of debugging down.

Now on to tarantella/SGD I re-installed it and run tarantella start and saw mysterious "Segment faults" in the start scripts, not sure what is going on haven't had time to debug them fully, but at least it runs somewhat.

Now to move on to coolstack, I reinstalled it even though all the binaries were imported with my data pool I did the reinstall because i realized that there is no way to reimport the SMF manifests for it, but still this shouldn't be a that big of an issue. That is what I thought anyway. After I did the install that appeared to go with out issues, I attempted to start up mysql, and it failed, I finally tracked down a log file with the error in logged in it, libssl.so.0.97 wasn't being found. Apparently /opt/coolstack/lib/libssl.* symlinked it from /usr/sfw/lib where openssl is no longer there. Okay I tried using some other copies of the libs i had from Blastwave.org still no go.

Webstack 1.4 is supposedly its replacement, I went and downloaded the packages and tried to run ./install, like the README said, it required me to figure out which modules I wanted, after much trial and error all I got was "WS015 This platform is not supported" after verifying I downloaded the correct architecture, and Solaris version I ended up installing the packages manually instead of the nice installer. After 6 hours of debugging, I'm still quite a bit of away from having the machine back to where it was this morning, still need to configure apache, and php.

What a pain what happened to binary compatibility, all this happened from the OS that is supposed to support binaries compiled 10 years ago, as of now it can't even handle binaries compiled a year ago. Hope they fix these issues.

update:
Given up on b115, because twice now while trying to get squid the included one to work I have gotten the system into a crash loop, activate it and it crashes the system and then i crashes the system as it comes back up, first I thought it was my fault in that i did something wrong, so I started over. But nope, twice now. So I'm currently downloading Sol10U7 and going to give it a try.

timf:
asked the question when was the last time xvm worked, and it worked fine in b86 that the system was running before i did this upgrade. Everything worked fine in that relaease. Zero issues, mostly just wanted to upgrade to keep current.

System Config
asus m2a-vm motherboard
amd x5200+ cpu ( dual core 2.5ghz 1MB l2 cache per core)
250 GB sata drive on board. (root pool)
4x 500GB sata drives (data pool) spread betwen the on board controller and a realtek add on board in AHCI mode.
6GB of ram

Gmail + Laptop Tip

2009-06-05T12:40:00.000-05:00

I am an owner of a 16” LCD laptop, while the screen is nice in most aspects, but they have a low resolution of 1366 by 768, which is low by today’s standards especially if you’re a geek like me and have at least a 19” screen on your desktop. The one application, well webapp, that I have a problem with on this system is GMAIL with the multiple inboxes (available from the Gmail lab) enabled. I have found a way to make it usable, everyone probably already knows it as I did, but just didn’t make the connection on how useful it would be. Press Ctrl-minus to reduce the size of the font used in the browser. And you can actually get enough text in each of the inboxes.

Two movies one day

2009-06-02T22:55:00.000-05:00

Saw “UP 3D” really good movie, not up to Pixar’s normal standard, but Still worth going too. It does suck that most theaters are charging an up charge despite their normal high prices, compounded with expensive food prices.
Hung around and saw Star Trek, really funny, and provided a history to the characters and the scenery didn’t seem too far from the original’s TV show’s décor which I was afraid of.

So two good movies to go and see, just remember always eat before the film, even if it means hitting McDonalds before hand, so you can get out of theater for less than $20 a person.

Google Wave

2009-05-30T22:08:00.001-05:00

checked out and signed up wave.google.com looks really cool can't wait to try it out my self

Technology older than I am can surf the Web

2009-05-29T13:09:00.003-05:00

Here is an interesting video and webpage on using a 1964 300 baud modem to get on the web.

http://hackaday.com/2009/05/27/1964-300baud-modem-surfs-the-web/#comments

Solve it all – during a BBQ

2009-05-25T13:14:00.003-05:00

Having spent the weekend with extended family, you always run into the questions and suggestion, you know the type, oh cousin blah needs someone to help with his company, he does websites, you do computer stuff, it should be a match made in heaven they all think. Then you get to break it to them sorry I don’t have any artistic ability so I won’t be any help in the cousins business. But he has a lot of work, he has 40 websites on his server, sure you can help. They just don’t get it; I’m the one that maintains 250 servers and designs future deployments that are then sliced and diced into servers/zones/containers/virtuals that are then sold to the hundreds or even thousands of smaller customers like the cousin’s web hosting business. Yes I can write programs in a dozen or more programming languages, but do I want to do it 40 hours a week, do I want to do it on a daily basis? Hell no. I struggled to create a few web pages for the home intranet that keeps a family shopping list and pulls in a couple RSS feeds. But that is the extent of my hobby, I’m not good enough to do it for a living, nor do I want to.

Of course this is followed up by requests for virus removals, solve disk space issues, what computer should I buy, should I retire my Intel P3 600 machine, and the list goes on, they all revolve around Windows. They all think you can solve everyone problems. They just don’t get it, yes I use and manage computers for a living, I know windows, and do I want to solve all their problems? Nope. At a position I held a couple years ago with a fortune 500 company, I didn’t even have Administrator rights on the company issued laptop, it was great, when the issues came up and they did I got to call the corp help desk and they solved them all. I could have had Administrator rights if I filled out one form, but I didn’t want them. I had root on 175 Solaris boxes and a dozen Red Hat boxes, that was enough.

For many of the problems I will be hitting Google. Then comes the others that think because I’m between contracts they are going help me out and actually pay. But again they don’t get it, as a UNIX admin taking money from them sets up a bad pattern that I just don’t want to start. It all starts out nice and innocently, I do some small job for them at what to them seems like decent money, but compared to what UNIX administrators earn, it’s nothing. I set up my life to be able to cope with the loss of my regular income for periods of time when I don’t have my typical paycheck. But the family just doesn’t get it so it leads into them wanting to help me and get there computer problems solved but once the pattern starts it leads to a few more jobs that pay next to nothing when they give my contact information to a few friends, etc all with good intentions of trying to help out. So you end up making next to nothing, doing work you don’t like and frankly are not very good at, and that is the best scenario by the end of the arrangement I end up putting more time into solving their problems than they or even I dreamed it would take, and I end up making even less money because how can I charge them for 100 hours for what they thought would take 2 hours, and I thought would take 5 hours max and quoted for 10 hours just to be on the safe side. In the past before I learned my lesson I ended up making less than a $1 an hour for hundreds of hours of work, sure it was supposed to be a stupid database application or automated form that I thought would take 5 or 6 hours that exploded into something way bigger with lots and lots of tiny details that required more and more time.

I’m being followed

2009-05-25T06:00:00.002-05:00

Yesterday I finally joined Twitter, yes I may be the last in the Social network age to join twitter, yes from time to time, I feel like I’m in information overload. But it may be a better way to follow friends. Facebook seems okay but I seem to be a constant flow or requests to try some strange application or your friend has gotten a new farm animal, find your redneck name, etc. perhaps twitter will be better just giving short snippets of what people are doing and searching for content.

Much to my surprise I already have 6 followers and it’s a holiday weekend.

If anyone else wants to follow me here are my "social network profiles, feel free to follow me or add me as a friend, and I will do the same. Perhaps later I will even get back to Solaris/UNIX related posts.

Facebook:

http://www.facebook.com/profile.php?id=557067672

LinkedIn:

http://www.linkedin.com/in/jamesdwisconsin

Twitter:

jamesdwi

Back Home - Avoid Thrifty Car Rentals.

2009-05-22T13:12:00.003-05:00

I got back yesterday, had a great trip, thanks to all those that sent condolences and prayers they were much appreciated.

Trip was great got to spend a lot of time with my daughter and connecting with my family.

The one bad spot was caused by Thrifty rent a car, at Orange County Airport. Avoid them at all costs they don't have a heart and refuse to work with people no wonder they have to charge less to get people to rent from them, I was trying to pick up the rent a car that was already paid for, and they ran a credit check but a glitch in there system caused the check to fail, we even had Expedia, who went above and beyond for us, called and they refused to pick up there phone, only when we gave them our cellphone would they talk to expedia. Still they refused to help us causing us a delay of a 1/2 an hour. We ended up going to Budget who ran the same credit check and rented us a car.

RIP Mom

2009-05-15T09:14:00.003-05:00

Well my Mom took a turn for the worse, and died at 3:43pm on Wednesday, while I was still in the Air thanks to my younger brother calling me on Mother's day from the hospital, I was able to talk to Mom and say "I Love you", so am I am okay with not being any to see her before she passed.

Heading to the OC

2009-05-13T09:25:00.002-05:00

Flying to Orange County, CA today, wish it was for a better reason, my Mom is in the hospital.

But is currently stable, but they are aren't expecting her to make it. :-(

Open Office 3.1 is released

2009-05-07T18:59:00.003-05:00

I have been using OpenOffice for home stuff for years and it keeps getting better, perhaps it will get even more use by others because of the economy to excellerate acceptance. I'm downloading the latest version now to give it a try it appears to have some nice new features.

Download it here

Simple tests of my home ZFS server

2009-05-06T10:25:00.003-05:00

Some simple tests on my home ZFS server, it's running an older version of SXCE, only b96, current is b113, But thought my readers would like to see what ZFS can do on poormans hardware, Dual core x5200+ with the 1MB per core l2 cache, asus m2a-vm motherboard 6GB of ram and 4x sata 500GB drives...

amd:test# psrinfo -v

Status of virtual processor 0 as of: 05/06/2009 10:28:00

on-line since 05/05/2009 15:10:02.

The i386 processor operates at 2600 MHz,

and has an i387 compatible floating point processor.

Status of virtual processor 1 as of: 05/06/2009 10:28:00

on-line since 05/05/2009 15:10:09.

The i386 processor operates at 2600 MHz,

and has an i387 compatible floating point processor.

pool: tank

state: ONLINE

scrub: none requested

config:

NAME STATE READ WRITE CKSUM

tank ONLINE 0 0 0

raidz1 ONLINE 0 0 0

c0t3d0 ONLINE 0 0 0

c3t3d0 ONLINE 0 0 0

c0t2d0 ONLINE 0 0 0

c3t0d0 ONLINE 0 0 0

time (mkfile 100g test ; rm test)

real 26m16.079s

user 0m1.678s

sys 1m47.347s

amd:test# zfs set compression=on tank/test ; time (mkfile 100g test ; rm test)

real 12m19.286s

user 0m1.136s

sys 0m57.637s

Automate your home server

2009-05-05T09:26:00.001-05:00

At work we all think nothing about automating tasks, want a daily report put in a crontab, backup some files create a netbackup job, remove old log files another script, sure this is all great, but when was the last time you did this at home?

Most of us at home we take backups of our home directories when we think about it, if we are really bored or make a large change to our database and we are worried about it we take a dump of our database. But what about the little changes we make all the time or a link to our home intranet sure its not important but do you really want to recreate it?

I just yesterday did an dist-upgrade to my Mythtv box, and it didn’t go well, it has only been 3 years since I installed it, and never thought of doing a dist-upgrade, but it wasn’t broken, but of course it broke but it’s really my fault, I booted up a live disk, and backed up my home directories and /etc , and proceeded to try to do an upgrade, but it failed, Of course the rebuild process went well. When I get around to restoring my intranet, I remember oops forgot to save a copy of the data files. Do I blame Linux? No It’s my fault, it told me to backup the files before trying to upgrade, and I was really just a bad hard drive away from failure anyway, since it’s a home box and /var was sitting on a non mirrored disk. Well I am now restoring the data from my last database dump. Oh well it was from February, probably a couple dozen links are missing and the current shopping list.

So the moral of our story, automate stuff like backing up our home directories to a second machine, we harddisk is cheap we all have at least 2 machines, even if we just backup the data to another desktop, it only takes a few commands in a script to copy them over, dumping Mysql databases to an SQL file and scp it to another machine is easy enough and put these tasks to your crontab.

I am now going to move the database to my Solaris/ZFS fileserver and putting the data on ZFS so I can take snapshots and survive the loss of a disk and setting up automated snapshots the question now is do I create a script to shutdown Mysql take a daily snapshot, and then restart it just to be paranoid about the data files being out of corrupted during a database writes.

Wonders if any of my readers have "Hamthrax"

2009-05-02T22:40:00.001-05:00

So far me and my family are all swine flu free... And yes we all still eat pork products, since its not how its spread. Comment if you or your family has it.

OpenSolaris town hall

2009-04-24T09:55:00.001-05:00

I'm online now.. starts at 10am cst, or 5 minutes from now.. lasts for 2 hours.. so get in now or check for recording of it later link below with international call in numbers.

http://wiki.genunix.org/wiki/index.php/OpenSolaris_Townhall_Apr_24_2009

it can have 4096 cpus, it scales right?

2009-04-22T09:30:00.005-05:00

A commenter on my previous entry wants to say that Linux Is scalable because it supports 4096 max CPUs, but it is just a number it takes more to scale than just to allow the existence of CPU’s in the kernel. Take for example if you have a lot of CPUs, one will fail from time to time, right? How do I remove one in Linux? Let’s ask Google.

http://mailman.lug.org.uk/pipermail/nottingham/2008-March/010937.html
On Tue, 2008-03-04 at 13:37 +0000, Martin wrote:
> Does anyone know if a /proc or /sys tweak can be used to enable/disable
> a CPU or CPU core in a running Linux system?

echo 0 >> /sys/devices/system/cpu/cpu1/online

I didn't know that until I just Googled for it. It's a nice trick :)

Graeme

Is it just me or does poking magic numbers into a virtual file system to control the kernel make you nervous? Where is the man page for this process? It’s not on my Linux desktop box. What happens if the CPU can’t be off-lined for some reason, how do you script for this and check for failures, looks like you have to do some mighty deep black magic for that. Does the system support hot spare cpus? With Solaris it’s handled by psradm that returns a standard error code if the process fails and doesn’t return till the process is complete.

The psradm utility changes the operational status of processors. The legal states for the processor are on-line, off-line, spare, faulted, and no-intr.

Now how about adding one or more CPUs to a live system, adding more memory to a live system? Opps Google comes up empty on this one except for links that mention this is done in the bios, of course BIOS is not accessible when the system is up and running. Hmm yet top500.org seems to be full of large Linux systems. I guess its time to mention the top500 isn’t a listing of large computers anymore, it’s a listing of clusters made up mostly of 2-4 socket system so a maximum of 16cpu/core servers combined together.
#2 on the latest list shows

Cores Rmax(GFlops) Rpeak(GFlops) Nmax Nhalf
150152 1059000 1381400 4712799 0

150,152 cores guess they are not running a single kernel on that beast; it’s composed of 9000 separate servers. How does this show that Linux scales, it shows that the benchmark was able to run across this many systems using an network/interconnect frame work. These type of clusters really don’t do well with tasks that can’t scale horizontally like large database applications, at one company I worked with had a single system with 48 CPU/cores and 196GB of ram running multiple instances of oracle with over 100 mounted file systems with the majority of them seeing constant activity, this type of configuration is just not tested much on Linux, yet in the Sun/Solaris systems this is only a medium sized server.
Now let’s take a look at a large hardware configuration Solaris is maintained to run on. The fairly new Sun m9000 system for example, can have 64 four core CPU in the system, spread across 16 CPU boards, each of these boards are hot swappable, and can contain up to 256GB of ram each for those doing the math that is 4TB of ram per system. And the machine can be reconfigured live with no reboots, the large configuration is designed to be split in two or more machines known as domains, and this machine can be divided into 16 domains, and hardware can be moved from one domain to another as necessary, need more CPU’s in domain 2, add more, its all done with no reboots, and can be expanded to 288 PCI-x slots all of which are hot-swappable meaning no unplanned downtime. The process to do all this is fully documented. When you have 4TB of ram and 256 CPU’s in the system running multiple copies of Oracle and applications it can take an hour or more for the system to get back to 100% use. Yes Sun installs the standard Solaris version on this machine, the same one you can download and install on your home machine, the only difference is your home machine is most likely x86/x64, while that machine is UltraSparc, but if you do happen to have UltraSparc based machine to use, the same disk will install on your system with no changes, no kernel recompiles, no tweaking kernel configuration.
Simply put Solaris is designed to grow and shrink the size of its system online in real time and fully utilizes the system, each Solaris release is fully tested on largest and smallest systems.

What will happen to Solaris/OpenSolaris

2009-04-21T11:32:00.003-05:00

Well some of my readers are wondering what will happen to Solaris and OpenSolaris. Here are my opinion on the subject. Solaris isn't some technology that Oracle picked up by accident, Oracle knows all about Solaris and the people that Created and maintain it. Solaris was oracle's chosen platform for years because its so well built and maintained and runs Oracle so well. It was what Oracle was developed on. A few years back Linux was given that role because it was the most popular platform. I'm sure Solaris will be given the nod once again shortly since any professional developer and support staff can see the benefit of running on Solaris because of its stable ABI/API and its stability and scalability. There is something to be said about using the same code on a 1 cpu/1GB server and on a 512 CPU/ 1TB of ram server. Not to mention not having to worry about the next release of the OS break your code. I recently heard of an Admin taking Oracle 7 code and moving it to Solaris 10, note Oracle 8 was released in 1997, that is 12 years ago. Linux has issues runing code that was compiled for kernel 2.4.x. The only change the admin had to do to make Oracle run was having Solaris tell the application that it was running on Solaris 8, because Oracle 8 didn't like being told that it was running on Solaris 10. Solaris is not Oracle's first attempt and supplying an OS, it released unbreakable linux a few years ago, in an effort to get in to the OS game and to control the platform it is deployed on.

So my guess would be is that Oracle will keep Solaris around because its programmers and the platform as a whole make a perfect match for Solaris. OpenSolaris will be kept around as well to motivate Linux users to migrate to the Solaris platform away from Linux. Also there will probably be very few cuts in the Kernel/OS group of Sun so that Solaris maintains its current reputation of a stable system, its a lot harder to have a stable application if your OS isn't rock stable.

I should also mention that Solaris and Sparc hardware is the main part of Sun that makes the most money on, so I would guess that Sparc and Solaris is not going away. There is no money in x86 hardware except for some of the larger machines like the x4600 and maybe blades. The rest have a very low profitability.

Tips for Sun employees

2009-04-20T13:47:00.004-05:00

Be sure to create a project so that your semaphores don't exceed default values.

please create stored procedures for any activities you do on a frequent basis.

When you are working for Oracle and it seems that management just isn't listening, you contact the DBA's and have them restart the listerner.

Joke of the day

2009-04-20T08:25:00.002-05:00

oracle replaces "ls" with select * from directory where name="%" ;

i guess this shows i am not DBA, as Jason points out the correct syntax is

select * from directory where name like "%" ;

Oracle to buy Sun Microsystems

2009-04-20T08:01:00.004-05:00

This is good for both companies, Oracle and Sun have been long time friends, and Solaris was once its top teir OS and it will again. And Oracle will now own the full stack from the Application teir to the bare metal, need a SAP solution? Call oracle they will send you out a server, need a larger solution for your fortune 500 company? they will send you a blackbox(mini datacenter) fully loaded and configured to run your app,just plug in networking, power, and water. and set your company name and your ready to server 500,000 customers.

CNET: Oracle buys Sun: The big picture

New York Times: Oracle Agrees to Acquire Sun Microsystems

Reuters: INSTANT VIEW: Oracle offers to buy Sun Micro

Ten business lessons from 'Battlestar Galactica'

2009-04-12T19:25:00.001-05:00

Good for a laugh... Might even make say they are right.

http://www.macworld.com/article/139561/2009/03/bsg.html

the beginning of the end of opensource inovation.

2009-03-19T09:12:00.003-05:00

Surely you have all read the rumors, Sun wants to sell it self to IBM and IBM appears to be interested. But it sure is bad news for the OpenSource community as a whole. Take a look at where the innovation in opensource is coming from. Is it IBM? nope, Linux companies? sure there have been a handful of projects that are interesting but most of them are just clones and poor imatations what Sun has created, systemtap, btrfs the Linux croud will say that these add features.

Sun is the most innovative opensource developer, NFS, SMF, DTrace, ZFS, Crossbow, Zones, BrandZ, Java, Glassfish to name a few. Linux has lots of little stuff happening, mostly cloning and poor imitations of what Sun has created, and drivers for hardware which are nice but the age of Large Innovations in OpenSource is coming to a close if IBM buys Sun.

IBM donated an old filesystem/volume manager and some other tidbits of code. And all its bread and butter products stay closed off and stagnant. Seen any new improvements in AIX, DB2, Websphere, lately? Has any of it been opensourced? Sure IBM is an opensource supporter, it supports as long as its guaranteed to add to its bottom line. Surely IBM has some old tidbits that they could pass along to the linux community, VIOs, LPARs, and lots more in its source repository of the last 50 years of code it has written.

Has IBM offered the opensource community any use of its patents?

Sure IBM will now own all of Sun's opensource projects but they will be the first to go when its time to balance IBM budgets. IBM will rake in all the profits from support contracts and maintain Sun's products but will the changes be put back into the community? Nope, and before the opensource people say, "but they have to its the license", sorry if you own the product that is licensed you don't have to give back anything. 95% (yes I'm probably understating this by 4-5%) of all Solaris development is happening by Sun employees. Most community developers can only dream of understanding the lower level of the system, take a look at UFS code. ZFS is also some very complex code to understand as well. I guess the only thing we can be happy about is the fact that it is opensource, who knows maybe we can get some ex-sun employees to give time to the projects while the rest of the community gets up to speed. IBM is not going to invest the time to opensource most of Sun's code, what is opensource will stay opensource, but anything not there yet will stay locked up in IBM's source repositories like AIX, DB2, Websphere, etc. Change the license of ZFS probably won't happen unless IBM is feeling very charitable or its marketing folks feel it develops a LOT of goodwill and some major $$$ on its bottom line, IBM will do its best to keep ZFS quiet, and will begin singing the praise of the Linux's btrfs to replace it, eh its cheaper to give $10,000 to the Linux crowd than have there lawyers change the license on code they own you do realize how much IBM legal staff bills to review code right?

Blue suits and stuffy bushiness men don't get along with long beards and pony tails of the Open Source community, even Sun's dress code doesn't match well with IBM, Sun's dress code by the way is simple. "You must dress, use your best judgement." Perhaps all this is why Sun moved so fast to opensource all of there code... They knew if they showed enough potential or there stock price fell low enough some big company would come along and buy them. This could of been the plan from the start of Johatan's tenure as CEO, opensource it all, while Scott McNeally went around trying to get offers to buy the company, and Sun Management knows that most of Sun's people wont mesh well with the IBM's culture, and they will move on to start up companies around all the Code that Sun has released, Sun employees have already been responsible for at least a dozen highly profitable start-ups perhaps this will spawn a 100 more, and there may even be a golden parachute for Sun employees as part of the deal, reevaluating Sun options that many Sun employee hold, just change the ask price from $70 to $7, and there would be a nice going away present to Sun's long term employees, just to make the transition go smoother.

back from glug

2009-03-10T22:15:00.002-05:00

I physically went to my first OpenSolaris user group tonight. I was sure I would first attend an OpenSolaris usergroup meeting in Milwaukee, but the Solaris gods have spoken, thanks to my new contract gig in IL, I ended up having to skip the Milwaukee OpenSolaris meeting scheduled to happen next week and attend this one. It was a great meeting learn about some of the joys of live upgrade.

Jkstat rocks

2009-02-27T22:05:00.002-06:00

Cool new gadget with cool graphics. Am I becoming less jaded, do I just have too much CPU power at my disposal? Too much ram, after all I have 4 AMD cores and 10GB of ram spread between the two machines that sit in my living room?

If you told me a couple years ago would I be spreading the news about a cool new java application that simply puts a wrapper around a text interface, I would say you were crazy. But here I am today going to give you the news of one, jkstat. Peter Tribble has done a great job using java to display Solaris kstat information.

Here is a screen shot of one small part of it, it shows a graphical representation of my file server scrubbing its disk, there are a lot more goodies to it be sure and check out Peter's page it has a lot more pretty pictures, and I'm too lazy to take more of my own system.

While it's a great program I do see room for improvement, it would be great if he integrated the logic from Ben Rockwood's arc_summary into the ZFS section. A way to export the data being displayed in XML would be highly useful to use in executive reports and perhaps more graphic goodness using the office suite of your choice. Also it would seem quite simple to add the iostat -z option functionality to the iostat chart, it would be especially helpful on ZFS pools since all of the IO is limited to only one or two slices the rest is mostly 0's

how do the professional sysadmins do it

2009-02-26T18:16:00.004-06:00

Perhaps you have dreamed one day of being a System Administrator for a living? Perhaps you just have 5 or 6 boxes at home? Perhaps you are just bored and ended up reading my blog by accident.

Here is one of the secrets of the guys that administer 100's or 1000's of Unix systems. Just being the administrator of a couple dozen boxes can be hard without an network wide authentication system. Yet I have managed to fall into that role at 2 of my last 3 jobs. So its probably wise to learn how it can be done.

First step is doing a process called transferring keys to all the systems you administer. Google password less login ssh and you will find the steps, i'm base-ing this on Shortest passwordless login guide, but of course they are going to show you how to do it on a box or two. I'm going to show you some simple scripts that take it to the next level. I'm not going to automate the full process, because many places frown on automation of password entries.

create a list of your servers... one server per line in a simple text file, call the
file servers for this script to work.

#create your key
ssh-keygen -t dsa

#copy your new key out to all the servers, and make ssh use it.
#the mkdir below may fail if the directory exists, ignore the error its harmless
for i in $(cat servers) ; do
echo SERVER=$;
scp ~/.ssh/id_dsa.pub $i
ssh $i "mkdir .ssh ;
chown 700 .ssh ;
cat ~/id_dsa.pub >> ~/.ssh/authorized_keys ;
chmod 644 /.ssh/authorized_keys;"
done

that is the heart of it, the worse part is entering your password twice for each machine.

now here is a second script to test it.

for i in $(cat servers) ; do
ssh -o "BatchMode yes" $i || echo $i >> ServersToFix
done

now you have a list of servers that didn't log you in automaticly, the -o "BatchMode yes" as you might of guessed tells ssh to only use keys to log you in. If keys don't exists or something else is going on the machine is added to the ServersToFix list. that you can go through and fix.

Once you invest the effort to make this all work, you will be doing things on all your machines almost as fast as you could do them on one. This is how the professional sysadmins do it, and manage to keep security off there backs.

Now add a few more bells and whistles and you get this.

for i in $(cat servers) ; do
echo server=$i
ping $i &&
ssh -o "batchmode yes" $i "dmesg | tail -n 100"
done > server_error_logs

this grabs the list 100 lines from the system logs of the servers that are up, you can then review them while you enjoy your first cup of coffee in the morning and see what if anything bad happened to your machines recently.

Write a blog become famous ;-)

2009-02-24T20:28:00.002-06:00

Wish I could of been there in person, but Ben Rockwood did a great job telling the down and dirty truths of ZFS in the production environment. Check it out at ZFS in the Trenches: Presentation from OpenSolaris Storage Summit.

Pay special attention to the end of his segment and the slides and you will see my name which is pretty cool to see my name mentioned by the Solaris/Storage Rockstar Ben Rockwood. Thanks Ben.

the power of 3's

2009-02-19T20:30:00.002-06:00

Time for another post. I have started another contracting assignmnet, I'm going to keep posting as much as I can.

If you haven't noticed most of the new stuff in Solaris 10 has just 3 commands or less. Here are a few examples, not sure if it was part of an initiative inside sun or what. And I'm sure I have missed a few.

SVC
svccfg - configure manifest and services
svcadm - enable, disable, reset, reload sercices
svcs - list services under svc control

ZFS
zpool - configure, monitor, maintain your storage pool
zfs - manage, monitor, maintain your filesystems
zdb - get the hidden secret tidbits of your filesystems, hardly anyone use it

DTRACE
dtrace - it does it all.
lockstat - based on dtrace but is still just a glorified dtrace program

FMADM
fmadm - fault management configuration tool
fmstat - report fault management module statistics
fmdump - fault management log viewer

Wearing your heart on your sleve

2009-02-12T21:57:00.002-06:00

This story brings new meaning to wearing your heart out on your sleve and is highly cool, its amazing what scientists are doing.

Scientists turn skin cells into heart cells
By Mark Johnson of the Journal Sentinel

In work that extends science's power to alter the basic unit of a human being, researchers at the University of Wisconsin-Madison have taken skin cells that were reprogrammed back to their embryonic origin and grown them into functional, pulsing heart cells.

The code for my Sun Pictures page

2009-02-11T08:57:00.003-06:00

Here is the PHP code to the story 20000 words on OpenSolaris and Sun complete with the 2 collum layout.


<div id=left>
<?php
require('rss_lib/grabfeed.php');
print_feed('http://www.flickr.com/services/feeds/photos_public.gne?&tags=openSolaris&format=atom_03', 1, 1, 1, 0, 20,10,0,0);
?>
</div>
<div id=right>
<?php
print_feed('http://blogs.sun.com/main/feed/entries/atom', 1, 1, 1,0 ,20, 10, 0, 0);
?>
</div>

See I promised I would release the code. Happy now?

I guess you want a little more click below to see the rest. Grabfeed.php

OpenSolaris Sample Chapters

2009-02-10T10:11:00.005-06:00

Dave Miner one of the authors of the OpenSolaris Bible has released a couple chapters of the book in pdf form.

Crash Course, looks to be a good place to start for people coming to OpenSolaris from the Linux. and ZFS is the part that seems to be the main draw of Linux users to OpenSolaris these days.

Click on picture to go to Amazon to order its now $33.74 currently and includes free shipping.

is the Legacy too much?

2009-02-10T08:57:00.003-06:00

I read this great blog entry Solaris Spit & Polish, by Ben Rockwood. And agree with what he says.

What has always been something we perhaps ignored or downplayed has become far more starkly contrasted by truly easy to use yet complex things such as ZFS or SMF....
Sun is very good at engineering the big things, but I've noticed that when it comes to connecting all the dots they tend to turn toward the path of acquisition.

But I think what really is happening is that the new bread of Sun engineers were given the goal of making new things easier, and No one dares do work on the older stuff, like LDAP, the shell, and anything else that came to life before Solaris 10, for fear of breaking compatibility of someone script or application written 10 years ago. Just upgrading to ksh93 was a monstrous undertaking because it had to be verified to run all the old scripts.

But how do we fix this? Is it time for SunOS 6.0? For the hardcore folks they know that currently Solaris 10 and OpenSolaris report as being 5.10 and 5.11. Yes this is a radical idea, but I'm not saying to break the API or ABI, just giving OpenSolaris the ability to change things that may break scripts without a year long process to get it approved for OpenSolaris.

# uname -av
SunOS blade1000 5.11 snv_107 sun4u sparc SUNW,Sun-Blade-1000

How can we move into the 21st century with a ton baggage from the 20th Century? When you remodel your house or even just your kitchen do you use the same door knobs and switches? Let the community play.

How about some even more drastic ideas, how about putting SunOS 2.5.1 and Solaris 8 back up for download, no don't offer support, or patch it any more than it is today, Just make it available for download and say for hobbyist use only. So if some hobbyist or a kid in a 3rd world country finds him self the proud owner of a Sparcstation 5 or Sparcstation 20 or even a Ultra 1, he too can see how these Sun systems were supposed to be run, and not have to put Linux on it and watch them limp along, thinking and this is why they call it "Slowaris" Perhaps someone in the community will find a tiny nugget of gold in the old packages and think why did this die, its cool, lets do a retro version of OpenSolaris with an OpenLook user interface. No the mainstream is not going to want it but does it hurt to give the community a full toy box to play and draw ideas from? Face it the world is changing the next great developments are going to happen in the 3rd World, the 3rd world is huge, and is getting better educated than many here in the U.S.A. China and India together have almost 2 Billion people and with that many great minds some brilliant things are going to come from that part of the world.

Back in the 80's the key to getting Sun and Solaris into the future was putting Sun'd and Solaris in the Universities, today the world has changed lets get OpenSolaris and Sun hardware into the hands of the future. Put OpenSolaris in the University, and give the kids in the 3rd world a chance to see the glory of Unix as it was meant to be. No this won't help the kid out in an african village but it may help some kid that found an old retired computer get started on a path to great discoveries and may keep a few 100 tons of old Sun gear out of the landfills for another decade.

Kindle 2 is ready, but why?

2009-02-09T12:09:00.004-06:00

If you haven't heard all the buzz, Kindle 2 is now available for pre-sale, the question is why? It looks cool, and probably does a great job of reading books. The price is too high, $359 for a device that allows you to read books, sure the cell-wifi is nice but is limited to downloading books and magazines. For the same price you can have a new netbook that runs Linux and read books and do real work, includes wifi not to mention 1GB of ram and 160GB hard disk space, though it is limited to regular wifi networks not cellphone networks that the kindle is. The price is the same you decide which is the better deal?

Kindle 2

ASUS Eee PC 1000HA

OpenSolaris Bible now shipping

2009-02-09T10:23:00.003-06:00

Click on picture to go to Amazon to order its now $33.74 currently and includes free shipping.

You can see it has many advanced topics that are not really availible in other Solaris related books at least not in a single book.

Table of Contents

I. Introduction to OpenSolaris.

1. What Is OpenSolaris?

2. Installing OpenSolaris.

3. OpenSolaris Crash Course.

II. Using OpenSolaris

4. The Desktop.

5. Printers and Peripherals.

6. Software Management.

III. OpenSolaris File Systems, Networking, and Security.

7. Disks, Local File Systems, and the Volume Manager.

8. ZFS.

9. Networking.

10. Network File Systems and Directory Services.

11. Security.

IV. OpenSolaris Reliability, Availability, and Serviceability.

12. Fault Management.

13. Service Management.

14. Monitoring and Observability.

15. DTrace.

16. Clustering for High Availability.

V. OpenSolaris Virtualization.

17. Virtualization Overview.

18. Resource Management.

19. Zones.

20. xVM Hypervisor.

21. Logical Domains (LDoms).

22. VirtualBox.

VI. Developing and Deploying on OpenSolaris.

23. Deploying a Web Stack on OpenSolaris.

24. Developing on OpenSolaris.

CSS added to Sun/Solaris pictures

2009-02-08T11:40:00.002-06:00

I have added some simple css to the sunpics page that I mentioned at 20000 words on opensolaris and Sun.

I still have more work todo, some blogs use broken html, like <img src="test.jpg"width= note there is no space between the quotation mark and the width= field, I am working on this. I also want to put the images into a table so each feed shows up as a table then I won't need to use columns.

b107 is coming to a blade1000 near me

2009-02-07T13:23:00.003-06:00

I am currently installing SXCE build 107 on my Blade 1000, 2x 750mhz cpu's and 4GB of ram. I have just upgraded the second harddrive to 144GB, but of course it needs to be reformated, to 512byte sectors, but you couldn't beat the price.. $40 shipped for a 144GB fcal drive 10k rpm.

This will be my first ZFS root/boot system, I will let you all know how it goes, but i guess i have 4-5 hours of drive reformatting before I can continue.

Update:
B106 broke Sunray software, which of course I found out after 4 hours of banging my head against the wall, and upgrading to the latest 4.1 SRSS software and filing a bug report at bugs.opensolaris.org that will most likely be marked a duplicate tomorow. You can find the work around for this at Bug ID#6799655 Just modify the file mentioned and reboot the system, yes the reboot is necessary, utrestart -c is not good enough.

Other wise the new version of SXCE has lots of new goodies, and the installer is smooth, it even handles mirroring of the ZFS pool by just telling it to use 2 disks.

More updates to come.

20,000 words on OpenSolaris and Sun

2009-02-06T20:57:00.002-06:00

A picture is worth a 1000 words, but I don't have a graphic artist bone in my body, I don't like the idea of walking around with a camera strapped to my hip. Yet I want to have nice pictures on my Intranet and utilize my home lan network, what good is gigabit network back bone and gigahertz servers if I have no pretty pictures. As I talked about here, rss news feeds on the Intranet. I now have news articles on the Intranet, but I wanted more. I have now created a php code that strips out the image tags. from news feeds and displays them. I will post the code to this in the next few days.

For now you can check out the sun image page that demonstrates code.

The code really doesn't do much to pretty up the output, I am working on better format like putting the images into a table so they can be 3 or 4 images per line. If you like this comment here and I may keep it up and perhaps even motivated to do more.

6 more reasons to hate Windows 7

2009-02-04T18:03:00.002-06:00

PCWorld broke the news that there will be Six Editions of windows 7.

Want Aero effects, Media Center, and the ability to run more than three applications at once? Then you can't settle for Starter, which comes bundled on some new hardware. Instead you'll have to pay for Home Premium. If you want those plus more presentation tricks and other extras, cough up the cash for Professional.

I guess I will need to upgrade to professional if I ever install it. 3 applications only running at a time, that is a bad joke on PC users, but I guess it will limit viruses, when you get 3 viriuses no other application will run. Three applications only is a step back to the MS DOS days, when we could run more than 3 applications at once thanks to terminate and stay resident applications, sidekick (my favorite application from those days.), as well as a clock application that sat in the upper right hand of the screen. Linux will have an even bigger chance to capture the desktop market.

My php Database class

2009-02-03T23:16:00.003-06:00

This is the second part of my PHP library/classes that I am using for my intranet. I am not a PHP coder, I am a system admin, so my stuff is not ready for production use, but may help you get started coding PHP, your millage may vary, no guarantees to the quality of the code or its security, if it blows up your house or destroys your computer, don't come after me. It uses the code in my_debug_library.

click below to see the code.
my_db.php

Football Insanity

2009-02-01T11:53:00.000-06:00

No this isn't another Superbowl post, well not exacty, I asked my wife this morning what time the super bowl was on today, and she switched over to NBC, and saw “road to the superbowl” 11:00-noon.And assumed the game would start at Noon. Nope that is not the case. I checked out the full TV listing for NBC and found the pre-game show came on at noon, so the game starts at 1 or 2 pm right? NO! the game starts at 5P.M. central time. So 5 hour pre-game show plus 1 hour pre-pre-game show. I like football as much as the next guy, okay my Green Bay Packers didn't make the playoffs. But I do intend to watch the game, but 6 whole hours of pre-game show? Is totally excessive.

I think they can show all the highlights of the previous 40 super bowls in 5 hours and throw in all the highlights of Bret Farve's 15 year career (did I mention I was a Packer fan?) as well in 6 hours.

Enjoy the game or what ever you are going to do instead... hey you have 5 more hours of pregame left you can go and reinstall Solaris 10 on your system in the time you have before the game. ;-)

Multi-Vendor iscsi tips and help

2009-01-31T08:05:00.001-06:00

I found this very interesting article on how to improve iscsi performance with vmware and a lot of the information probably translate to improving performance for all iscsi tasks. I have skimmed it, it looks very well done with tons of information, I will read it again once coffee is on board, it is early on saturday morning and my brain is only firing on one cylinder.

a multivendor post to help our mutual iscsi customers using vmware

RSS news feeds on the Intranet

2009-01-30T20:49:00.002-06:00

I have been doing a lot of work on the home intranet, with 2 desktops and 2 sunray's, I think its nice to have a place an intranet, with the basic information, so far its mostly accessed for local temperture and a shopping list. But thanks to Magpie RSS parser. I now can add more content to intranet without having to create it my self. Magpie is really cool and handles Atom and RSS feeds with very little code changes. And can be made to handle feeds that don't quite follow the standards.

I have added a news ticker with the latest local news and Green Bay packers news using javascript that I found on the web, I'm not happy with the ticker code I have and am looking for a replacement.

I also went out to find more feeds to include on the site, and found this Google help page with links to RSS for Top news, US news as well as bunch of countries.

At the end of page there is a really cool feature that allows to create rss feeds for any keyword you like. I of course created a news feed for the latest Solaris news.

http://news.google.com/news?q=solaris&output=rss

My simple php debugging library.

2009-01-27T07:41:00.003-06:00

This is the start of a series of entries that focus on PHP coding, this code is probably not ready for production but it seems to work well enough for my intranet site. Feedback is welcome and appreciated. I am not a programmer. I am a system administrator.

This is a small libary I have developed to help me write code, its nice in that you can turn on debugging information when needed with little or no changes to your code path, it is a library that all my other code depends on. In the next couple days I will post my database library, that makes it easy to access data in a mysql database and display it.


<?php
// Debug Library
//
// Change Log
//              Fixed some performance issues
//              Added the ability to call ddump with debugging enabled just for

global $DEBUGGING;
//$DEBUGGING=1; // uncomment this or add into code you wish to debug

function d_debug() {
   global $DEBUGGING;
   return $DEBUGGING;
}
//
// print a string if we are debugging
//
function dprint($string) {
   global $DEBUGGING;
   if(1==$DEBUGGING) echo  '<PRE>' , $string , "</PRE>\n";
}


// print a variablename = contents
function dprintv($string,$var) {
   global $DEBUGGING;
   if(1==$DEBUGGING) echo '<PRE>', $string, ' = ', $var, "</PRE>\n";
}
//
// dump the contents of a variable
//
function ddump($var,$debug=0) {
   global $DEBUGGING;
       if(1==$DEBUGGING or 1== $debug) {
       echo '<PRE><BOLD><font size=+2>Variable dump:</font>' , "\n";
       var_dump ($var);
       echo '</BOLD></PRE>' , "\n";
   }

}
?>

DTV delayed - updated

2009-01-27T07:37:00.006-06:00

Update: congress tossed the bill out, looks like the conversion back on schedule for next month :-)

I can't believe they have decided to delay the Digital TV switch over, siting that people aren't ready. People are never going to get ready if they keep delaying it. Some people won't get the converter box until they have no choice, like the day after analog TV goes black. I guess the years of planning and announcements is not enough.

The Obama transition team was alarmed, mainly because many people who rely on antennas are poor, minorities or elderly. The Nielsen Co. estimated that more than 6.5 million households -- or 5.7% of TV homes -- were not prepared to receive digital signals.

5.7% of people are procrastinators seem quite reasonable, I would of thought there were more procrastinators than that, so I'm really supprised the number is this low.

Obama's proposed $825-billion stimulus legislation includes $650 million to help replenish the program.

I guess the bailout bill is turning into the a huge pork barrel legislation, not that I'm against giving out more coupons, after all FCC made billions on reselling old analog frequencies, But it would of made sense to tie the additional money for the coupons to the bill delaying the Digital TV switch over. But I guess its asking too much for government to actually make sense.

But all this delays gives us geeks more time to build our mythtv systesms. If you are having issues with the digital signal and I highly recommend you get a new digital TV antenae, yes I was dubious of the idea of them, but once I did try one out I was amazed at how much the reception was improved.

Here is a nice cheap one, only $24.99 add another item from amazon sold item and get free shipping.

If you are made of money, here is the BMW of antenna's.

What I reading.

2009-01-24T07:41:00.003-06:00

Fiction

Inkheart

Non-Fiction

Solaris Internals

1.8 million people in Lego

2009-01-22T09:04:00.003-06:00

Couldn't make it to the presidential inauguration see it done in lego's at Legoland California.

VI love it or hate it, must learn it

2009-01-15T11:39:00.010-06:00

Okay I'm not vi's biggest fan, I have to admit. Yes I'm a hard core Solaris admin, and am supposed to be in love with vi, after all it was written by Bill Joy, one of the co-founders of Sun Microsystems. But after years of using vi you begin to pick up a few tricks, and I also read the book below. Whether you are a vi novice or an expert it will remind you of commands that you may have forgotten. Now even if you don't use vi to do your editing. The knowledge of vi can still help you most shells have a vi mode, in fact plain ksh is extremely hard to use on a daily basis without enabling vi mode, unless you are touch typist with 101% accuracy. You do know about vi mode in ksh right? If not run


set -o vi

and you will be able to do move around on the command line
^ beginning of the line
$ end of the line
j left one character
k right one character
9 go to line 9

and a whole lot more.

of course vi knowledge is embedded in shell scripting as well, sed, awk, perl, php, and even visual basic as well as just about any other modern programming language has commadnds that work with regrex which is a very powerful part of vi.

I was at my job last year and I was working with a coworker and my manager were in a datacenter editing some files in vi, in a UNIX based OS, and there was no other editor installed, so he was typing, even though I wasn't a big fan of the editor I was able to show him a few commands to speed up the process and both my Manager and the co-worker were really impressed we were able to complete our tasks a lot quicker because I had read the book below, just remember with this type of core knowledge UNIX book its not important to have the latest or greatest version, I always buy used books, and usually more than one, one for home, and one for the office it sucks not having the book when you need it.

Click to find a copy at Amazon.com

While you are there, get this book already if you don't have it, you can get these used for less than $1 each, usually much less plus $3.95 shipping. Yes shipping sucks, but $5 for tech books you really can't beet the deal and it will enhance your career and even in these tough times your manager won't balk at approving a $5 book purchase?

Colder than Santa's Village

2009-01-15T08:19:00.003-06:00

In Santa's Village its a toasty
-6°F

Current: Clear
Wind: W at 2 mph
Humidity: 84%
Rovaniemi is the hometown of Santa Claus and it is christmassy all year round

Weather for Milwaukee, WI
-14°F

Current: Mostly Cloudy
Wind: W at 7 mph
Humidity: 96%

Netbeans 6.5 review from a PHP hack

2009-01-14T15:55:00.005-06:00

First off I'm not a Java coder in any way. So in the past Netbeans 6.5 held very little appeal for me. I am a php part time coder, mostly for little simple sites, after all I am a System Administrator not a programmer.

But now Netbeans 6.5 has PHP support and its pretty smooth, it still needs some improvements it would be nice if tidy or a php/html indent aware program came included by default. I really do like the CSS module as well as PHP aware editor both helped me find some bugs in my code that I wouldn't of noticed. The CSS module also allows you to see an example of how your CSS will display the data and gives you drop down lists so you don't have to remember what option is. It was nice how you could drill down from a function call to the function declaration or even from an include/require statement to the code, and when click on a variable or function it automatically highlights other references so you can see how it changes, or for the amateur coder you can see that you have a typo in one instance of variable or function when the others aren't highlighted. I really liked the variable/function auto completion even in PHP mode, which seems usually reserved for C/C++ and Java in other ide's I have used.

I find the jMaki ( a javascript framework) integration interesting it may allow me to do some cool stuff on my home Intranet without researching every little thing to make javascript do some fancy things. By doing little more than drag and drop.

I also like the intergration of the database explorer than even works with mysql that I use for my small projects at home. Its not ideal but its far better than most other interfaces that I have seen in IDE at least free ones. I don't intend on paying for something that I use at home to create some simple web pages, it really is nice to see something that is free and community supported that works so well and meets my needs.

For previewing your code I especially like how they understand that your web pages may be on a local server and not force me to configure ftp to upload the files. It would also be nice if you could set a url for each file in the project so you wouldn't have to start from the main page and edit the url or drill down to the web page you want to work on.

Once you put your project under merquial control or any of the code versioning software you can see old and current versions of your code on the screen at the same time along with a color coded diff to see changes.

There is still a lot of room for improvements, like the ones I mentioned above, a php/html aware indentation formatter, a per file, URL setting in projects, as well as HTML preview mode that shows the output of HTML code without loading a browser, and possibly adding HTML WYSIWYG preview tabs. It would also be cool to be able to modify the data in database view and table displays.

But I do highly recommend that all PHP coders give it a try at least those that don't beleve that vi is the best editor for all programming needs.

First Policitical Post

2009-01-11T10:27:00.003-06:00

I have so far avoided posting on policitical posts in my blog and I promise that it won't become a pattern, but I saw the below link and thought for sure I would disagree with it, but I actually totally agree with what it says.
Why Barack Obama Needs To Fail

Fail is too strong of word, but of course its just to draw attention to the post. He does need to make mistakes. He is human, and Humans seem to achieve there best by learning from adversary.

So my advice to Obama is "Take chances, make mistakes, and get messy!“ After all it would be pretty hard to screw up worse than his predecessor.

P.S. extra credit to the first person that posts a comment from what TV show the quote is from.

Update: Martin got it, congrats