Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Thursday, May 13, 2010

Installing SnapLogic Community Edition 2.1.0 on Ubuntu 8.10 in 5 minutes

If you read my last post, you would know that I hit a snag running the community edition of SnapLogic 2.2. There appears to be a hard dependency on a license check (see ticket 2449). Since I have no idea how long that will take to resolve, I decided to find a way around the problem.

I chose to roll back to version 2.1.0. But in doing so, I had some struggles. SnapLogic 2.1.0 is based on Python 2.5.x and related components. My Ubuntu 10.04 install uses Python 2.6.x, and the 2.5.x stuff has been removed from the Ubuntu repositories.

After trying some minor hackery to get P2.5.x on U10.04 (building stuff from source), I gave up at trying to get the dependencies resolved. Instead, I just plopped down a new Ubuntu 8.10 VM on my machine, and got things going pretty quickly.

These are my notes. By the way, the 5 minutes I reference in the title is just for the SnapLogic install. Installing and updating Ubuntu takes about an hour, depending on network speeds.

  • Install 8.10
    • Download a VDI and install into VirtualBox, or
    • Do a bare metal install (e.g. dual boot)
  • Update your OS
    • 368 updates and counting as of today
    • Make sure not to upgrade Python beyond 2.5.x
    • Reboot
  • Download SnapLogic 2.1.0
  • Launch the SnapLogic installer
    • chmod u+x [blahblah.bin]
    • It will inform you what is missing from prereqs
  • Install the missing prereqs
    • Launch Synaptic Package Manager, and install:
    • python2.5-dev
    • libmysqlclient15-dev
    • libxml2-dev
    • libxslt1-dev
  • Rerun SnapLogic installer
    • Accept all defaults
    • Password is password
    • Elect to start up the servers at the end
  • Install Flash Plugin for Firefox
  • Navigate to the Designer UI
    • http://[hostname]:8081/
    • Open the Leads_to_Prospects Pipeline
    • Select the Run tab
    • Run the pipeline (this is where SL 2.2 fails for me)
    • Success!

 

SLDesigner

Technorati Tags: ,,

Thursday, May 6, 2010

Example Linux startup service for Python, Java, Ruby, Perl, or Script as a daemon process in init.d

This blog post provides sample Bash code for converting a Python, Java, Ruby, Perl, C++, script, or any other program as a Linux service that will start running at system startup. It is a script that is deployed in /etc/init.d like many other common service programs.

Note that there may be language specific ways of doing what you want. For example, there is an evolving Python package that is supposed to do this for you. At this writing, that package seems to be still in progress.

This solution has three components:

  • The actual program that needs to run as a service. In this example, it is a python (.py) program. But it really can be anything, including Java, Ruby, C++, or whatever
  • A wrapper Bash script that simply invokes the actual program. The only value add is that it writes the process identifier of the actual program to a file. You could possibly folder this functionality into the daemon itself, but I haven’t bothered to do that.
  • The daemon script. It is what is located in /etc/init.d and ultimately is responsible for starting and stopping the service.

Actual Program – echoHW.py

For this example, it does nothing real. It just spins the process:

import time

while 1 < 2:
    time.sleep(10)

Wrapper Script – echoHW.sh

This script simply invokes the actual program. It does also write out the process identifier to the file.

# Execute the actual python script
# The "$@" passes in any parameters into the python exectuable
# The '&' puts the process into background (as a daemon)
# The 'echo $! > mydaemon.pid'  write the process id to a file

python echoHW.py "$@" &
echo $! > mydaemon.pid

Daemon Script – mydaemon

This file must be copied into /etc/init.d.

#!/bin/bash
#
# mydaemon          Start/Stop any shell script
#
# chkconfig: 345 95 65
# description: mydaemon 
# processname: mydaemond
#

# ENVIRONMENT
# Edit these for your configuration

# Name for the service, used in logging
NAME=mydaemon

# Name of the user to be used to execute the service
SCRIPT_USER=plaird

# Example of how to pass paramters into the command
PARAM2=whatever

# In which directory is the shell script that this service will execute
MYDAEMON_SCRIPTS_DIR=/home/plaird/dev/linux

# Construct the command the will cd into the right directory, and invoke the script
MYDAEMON_COMMAND="cd $MYDAEMON_SCRIPTS_DIR; ./echoHW.sh 'param1' $PARAM2"

# How can the script be identified if it appears in a 'ps' command via grep?
#  Examples to use are 'java', 'python' etc.
MYDAEMON_PROCESS_TYPE=python

# Where to write the log file?
MYDAEMON_SVC_LOG_FILE=$MYDAEMON_SCRIPTS_DIR/mydaemon.log

# Where to write the process identifier - this is used to track if the service is already running
# Note: the script noted in the COMMAND must actually write this file
PID_FILE=$MYDAEMON_SCRIPTS_DIR/mydaemon.pid

# Load system specific optional arguments
# Create and populate this file with machine specific settings
if [ -f /etc/sysconfig/mydaemond ]; then
    . /etc/sysconfig/mydaemond
fi

# Is the service already running? If so, capture the process id
if [ -f $PID_FILE ]; then
    PID=`cat $PID_FILE`
else
    PID=""
fi

# SERVICE ENTRY POINTS (START/STOP)

# Start Command
start() {
    if [ "${PID}" != "" ]; then
        # Check to see if the /proc dir for this process exists
        if [ -a /proc/${PID} ]; then
            # check to make sure this is likely the running service
            ps aux | grep ${PID} | grep $MYDAEMON_PROCESS_TYPE >> /dev/null
            # If it is a process of the right type assume that it is mydaemon and just exit
            # otherwise remove the subsys lock file and start mydaemon
            if [ "$?" = "0" ]; then
                exit 1
            else
                echo "mydaemon lock file still exists, removing..."
                rm /var/lock/mydaemond
            fi
        else
            # The process running as pid $PID is not a process of the right type, remove subsys
            # lock and start mydaemon
            echo "mydaemon lock file still exists, removing..."
            rm /var/lock/mydaemond
        fi
    fi
    echo -n "Starting mydaemon: "   
    su - $SCRIPT_USER -c "/bin/sh -c \"$MYDAEMON_COMMAND > $MYDAEMON_SVC_LOG_FILE 2>&1\"" & RETVAL=$?
    sleep 3
    touch /var/lock/mydaemond
    exit 0
}

stop() {

    echo -n $"Stopping mydaemon: "
    if [ "${PID}" != "" ]; then
        echo -n "killing " $PID
        kill ${PID}
        for i in {1..30}
        do
            if [ -n "`ps aux | grep $MYDAEMON_PROCESS_TYPE | grep mydaemon `" ]; then
                sleep 1 # Still running, wait a second.
                echo -n .
            else
                # stopped
                rm -f /var/lock/mydaemond
                rm -f $PID_FILE
                echo
                exit 0
            fi
        done
    else
        echo "$NAME is not running"
        exit 1
    fi
    echo "Failed to stop in 30 seconds."
    kill -QUIT ${PID} # Request a thread dump so we can diagnose a hung shutdown
    exit 1
}

case "$1" in
  start)
      start
    ;;
  stop)
      stop
    ;;
  *)
    echo $"Usage: $0 {start|stop}"
    exit 1
esac

Installation

1. Start by copying the actual program and wrapper script somewhere on your machine in a permanent location.

2. Next, edit the mydaemon script to reflect the location where the program was installed. You may want to rename the file, and update the name of mydaemon inside the file.

3. Copy the mydameon script to /etc/init.d. As in:

sudo cp mydaemon /etc/init.d

4. Install the service. Installation will vary depending on your flavor of Linux. If you have chkconfig installed, it will be:

sudo chkconfig –-add mydaemon

If you are on a distro like Ubuntu, use update-rc.d, as in:

sudo update-rc.d mydaemon defaults

Service Usage

The service should now start on startup the computer. But you can invoke it manually as root:

sudo service mydaemon start

or:

sudo service mydaemon stop

Technorati Tags: ,,

Saturday, March 27, 2010

Solutions to VPN Problems with Ubuntu Linux – vpnc, network-manager, “terminated by dead peer”, WICD

I have been successfully using the Linux vpnc client to connect into the corporate network for months. However, I recently gained access to a more secure VPN network at the company, and I had nothing but trouble. This blog post explains the solution for that, as well as some other random network issues I was having.

Will this blog post solve MY problem?

I don’t know if I can help you, but here are the symptoms I was having on my machine:

  • I could establish VPN, but something truncated my connection every 5 minutes
  • After establishing VPN from certain locations, I could not ssh to my target instance; it never resolved the IP address
  • As I moved around the building (particularly up and down stairs), I had trouble with re-establishing network
  • I found the network hanging every 2 minutes for a few seconds

Troubleshooting

The key to solving these problems is to look into /var/log/syslog. Do you see either of these entries?

"connection terminated by dead peer detection”

This will be noted when something terminates your VPN connection"

“Roamed from BSSID 00:0F:61:1B:A4:32 (ESSID) to (none) ((none))”, followed by “Roamed from BSSID (none) ((none)) to 00:0F:61:1B:A4:32 (ESSID)”

This will happen every 2 minutes, and will be associated with a pause in your network traffic.

network-manager was the problem for me

I believe the source of all the trouble was the second error noted above. Based on Google searches, this indicates that the network card is scanning for new wireless networks every 2 minutes. During this time (which lasts 6 seconds) no other network traffic can be sent. It makes sense that this would cause issues with a picky VPN.

See this Ubuntu bug report for a full explanation (and confusion as to the proper fix) related to this nasty problem. Some folks indicated the network-manager component to be the source of the problem. network-manager is the default network manager for GNOME Linux distros (like Ubuntu). The problem reported in the bug report does not affect everybody – only some network cards. I run on a Dell Latitude E6400 with an Dell 1510 network card.

Sadly, the network-manager folks didn’t agree that it was their problem. So I had to look elsewhere for a solution.

WICD was the solution for me

Following guidance from others posting on the bug report, I uninstalled network-manager and installed an alterative called WICD (pronounced “wicked”). I have been running WICD for a week, and I am sold. All of the issues that I reported are now gone.

WICD has much of the same functionality of network-manager, but is a bit simpler which perhaps is its strength. Here is a screen shot of the WICD applet:

wicd_applet

If you wish to go this route (highly recommended!), issue the following command. It will install wicd and remove network-manager:

sudo aptitude install wicd

Note that wicd doesn't currently (as of 1.6.1) have menu driven support for VPNs. It is supposed to come with version 2.0 though, so keep checking in on this. It simply means that to start a VPN connection, you will have to do it from the command line instead of from the applet. If you are using vpnc, that means you have to do this from a terminal:

sudo vpnc

WICD Pro Tip

One thing that should be the default in WICD, but isn’t, is to always show the wired network connection in the list. Choose this by checking the box in the UI here:

  • Preferences->General->Always show wired interface