iMIS displays generic error when user attempts to download uploaded file

While uploading and linking to PDF files in RiSE with iMIS version 20.2.65.9955, I encountered an interesting bug, but I also identified a workaround. Today, I’ll share both the bug and the workaround here.

The particular page with which I was working uses the Content Collection Organizer iPart to display content from other content records within tabs. I observed that if I create a link to a PDF that has been uploaded in RiSE in the content record for one of the tab content areas, or subpages if you like, then publish the record, the website displays a generic error when I click the link to download the PDF:

An unexpected iMIS error has occurred. Please try your operation again; if you still receive an error, contact the system administrator.

That’s not very helpful, so I took a look at Event Viewer on the server and noted an HttpException with the following message:

Exception message: Failed to load viewstate. The control tree into which viewstate is being loaded must match the control tree that was used to save viewstate during the previous request. For example, when adding controls dynamically, the controls added during a post-back must match the type and position of the controls added during the initial request.

Interesting. Something’s happening in the iMIS/RiSE back-end code, then, which I can’t modify.

I did identify a workaround, however. If I create a download link in the main content record and publish that record, the links in the tabbed areas then function normally! Creating a standard link (e.g., with an href value of “#” or “/”) does not make this work correctly; the link must be in the format that RiSE uses when you link to a PDF that was uploaded to RiSE—i.e., with an href value like “javascript://[*]”.

The link apparently does not have to contain any text, however; it simply must exist. The presence of the following in the main page’s content record is sufficient:

<a href="javascript://[]"></a>

The link is not visible to the user because there’s no text, but it is the “magic sauce” that makes the PDF links within the tab content function as expected.

Using PHP and curl to post JSON data to the iMIS API

As a programming challenge, I recently decided to tackle using PHP and curl to connect to the iMIS API from outside the confines of RiSE. It’s relatively simple to get data from the iMIS API when you’re already logged in to an iMIS website, but I wanted to figure out how to post data to the API from an entirely different server. Documentation refers to this as direct access.

For my experiment, I created a PHP file on an external server. From a webpage within an instance of iMIS, I posted JSON data to my PHP file, which in turn retrieved an authorization token from iMIS and then used that token to submit the data to to the API.

<?php


// full URL of iMIS site
$url = "https://www.example.org";

// iMIS user's credentials
$username = "testuser";
$password = "testpassword";


if ($_SERVER["REQUEST_METHOD"] == "POST") {

    // JSON submitted by POST
    $json = file_get_contents("php://input");
    
    // ensure API URL and JSON are defined
    if ($_REQUEST["url"] != null && $json != null) {
    
        // address from which we get a token
        $tokenURL = $url . "/token";
        // API address to which we post data
        $apiURL = $url . "/api" . $_REQUEST["url"];
        
        callAPI($tokenURL, $username, $password, $apiURL, $json);
    } else {
    
        header("HTTP/1.0 401 Bad Request");
        
    $html = <<<EOT
<!DOCTYPE html>
<html lang="en-US">
    <head>
        <meta charset="utf-8">
        <title>401 Bad Request</title>
    </head>
    <body>
        <p>401 Bad Request</p>
    </body>
</html>
EOT;
        
        echo $html;
    }
}


// used to pass Ajax call to API
function callAPI($thisTokenURL, $thisUsername, $thisPassword, $thisAPIURL, $thisJSON) {

    // grab an authorization token to send to API with POST
    $token = getToken($thisTokenURL, $thisUsername, $thisPassword);
    
    // token length will be this short only if an HTTP error status code was returned
    if (strlen($token) < 5) {
        header("HTTP/1.0 " . $token);
    } else {
    
        // this is the header we will send to API
        $header = array("authorization: Bearer " . $token, "Content-Type: application/json");
        
        // initiate curl instance
        $curl = curl_init();
        
        curl_setopt_array($curl, array(
            CURLOPT_URL => $thisAPIURL,
            CURLOPT_HTTPHEADER => $header,
            CURLOPT_SSL_VERIFYPEER => false,
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_POST => true,
            CURLOPT_POSTFIELDS => $thisJSON,
            CURLOPT_FAILONERROR => true
        ));
        
        $response = curl_exec($curl);
        
        // tell browser the result of the call
        header("HTTP/1.0 " . curl_getinfo($curl, CURLINFO_RESPONSE_CODE));
        
        curl_close($curl);
        
        return;
    }
}


// retrieve token for use in API call
function getToken($thisTokenURL, $thisUsername, $thisPassword) {

    // this is the username and password we will send
    $content = "grant_type=password&username=$thisUsername&password=$thisPassword";
    // this is the header we will send
    $header = array("Content-Type: application/x-www-form-urlencoded");
    
    $curl = curl_init();
    
    curl_setopt_array($curl, array(
        CURLOPT_URL => $thisTokenURL,
        CURLOPT_HTTPHEADER => $header,
        CURLOPT_SSL_VERIFYPEER => false,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_POST => true,
        CURLOPT_POSTFIELDS => $content,
        CURLOPT_FAILONERROR => true
    ));
    
    $response = curl_exec($curl);
    
    $json = null;
    $returnStr = "";
    
    // return HTTP status code if there was an error; otherwise, return token
    if (curl_errno($curl)) {
        $returnStr = curl_getinfo($curl, CURLINFO_RESPONSE_CODE);
    }
    else {
        $json = json_decode($response, true);
        $returnStr = $json["access_token"];
    }
    
    curl_close($curl);
    
    return $returnStr;
}


?>

Naturally, you wouldn’t use something unsecured like this in a production environment; with the iMIS credentials pre-populated, anyone who hit the page could submit data to the API with no questions asked! Definitely a no-go. In addition, this PHP code retrives a new token every time it runs; that token should be saved and re-used until it expires.

Nevertheless, figuring out how to make this work was an interesting exercise, and I was able to connect to the iMIS API from outside the confines of RiSE. Such knowledge could come in handy somewhere down the road.

Very basic network troubleshooting

In a former technical support role, my colleagues and I received numerous calls and emails from customers regarding our network-connected devices that “weren’t communicating.” In some cases, those customers had legitimate complaints: network cards occasionally needed to be rebooted, and every once in a while an Ethernet adapter would actually fail completely and need to be replaced.

It was at least as common, however—and I think I could make a strong argument that it was more common—for the connectivity problems to not be related to my employer’s hardware at all! I wouldn’t necessarily expect the average user to perform network troubleshooting, but having to almost beg IT staff with some organizations to check their own network and their own equipment got old in a hurry.

With that in mind, I thought it would be worthwhile to compile a few basic troubleshooting tips that often helped me and the customers I was supporting determine whose team really needed to look into the problem. This is not intended to be a complete list of potential problems, but if you are new to technical support, or even if you are simply an end user trying to figure out whom you need to contact, these things may get you pointed in the right direction.

#1: Is the device turned on?

This one is so obvious that I almost hate to even ask the question, but seriously: is the device turned on? Are you sure it’s turned on? If someone disconnected the power adapter, or if your electrician flipped the circuit breaker so he or she could work on an electrical issue, and the device in question is powered off, you’re not going to be able to communicate with it.

#2: Is the device connected to the network?

I could just as easily have made this #1. Again, I hate to ask the question, but if we’re talking about a hard-wired device, does it have an Ethernet cable connected to it? Is the other end of the Ethernet cable connected to anything? Are there any Ethernet cables hanging loose at the nearest network switch?

Likewise, if the device in question connects to your network via Wi-Fi, does it actually show as being connected? Can you even see the network’s SSID if you take a quick peek at your phone?

In either case, if the device is not connected to the network, either physically or via Wi-Fi, you’re not going to be able to communicate with it.

#3: Are you able to ping the device?

Assuming that you’ve already checked the first two items—and you did confirm that the device is powered on and connected to your network, right?—my next recommendation is to try pinging its IP address from another computer on your network.

If you’re using Windows, you can open a command prompt by pressing Windows + R, then entering cmd and clicking OK. In the command prompt window, type ping 10.10.10.10, replacing the IP address with your device’s IP address, and then press Enter. You’ll likely find one of the following:

  • If you get a response with time values, then something with that IP address is connected, and we can troubleshoot from there.
  • If you get a response indicating that the request timed out, there could be a problem with the device in question, or there could be an issue elsewhere on the network. Proceed to item #4. (Note that some devices are configured to not respond to ping, so the lack of a response here may not necessarily indicate a lack of network connectivity.)
  • If you get a response indicating the destination host unreachable, there is probably a network issue that your network staff will have to investigate. Proceed to item #4.
  • If you get a message stating that the TTL expired in transit, a network device is misconfigured, and your network staff will have to investigate. Proceed to item #4.

#4: What does tracert show?

Using the command prompt window that you opened previously, try entering tracert 10.10.10.10, once again replacing the IP address with your device’s IP address, and then press Enter.

Depending on your network, you may initially see IP addresses or sever names along with response times listed in milliseconds, but eventually you will probably see asterisks along with the message, “Request timed out.” Provide your network staff with the last IP address listed with response times, which is the last network device from which your computer got a response, and that may help them narrow down where the problem lies.

One exception to this is when there is a network misconfiguration. In that case, you may see the same pair or sequence of IP addresses repeated over and over again. Even if that’s the case, you’ll still need to send the information on to your IT staff for further investigation.

#5: Does a different device connected to the same Ethernet cable as the device you’re troubleshooting have network connectivity?

One other thing you can do is configure a laptop to use the same network settings (IP address, subnet mask, and default gateway) as the device that you’re troubleshooting, then disconnect the Ethernet cable from the problem device and connect the cable to your laptop. If your laptop has network connectivity, you’ve confirmed that the physical connection itself is good.

I should add that not having network connectivity at this point doesn’t necessarily mean that there’s a network-related problem. Depending on firewall and switch configurations, and whether or not your IT team is doing any sort of MAC filtering, it may be impossible to connect your laptop to the network in this way, but if it does work, then you can rule out the network being the problem.

Wrapping it up

Again, this is not an exhaustive list, but simply a few questions that I’ve commonly asked when attempting to troubleshoot problems with devices not communicating over a customer’s network.

Once you’ve worked your way through this list, if you still haven’t identified the problem, then it’s time to escalate the issue to your IT staff or the support team for the device in question. Doing these few basic checks first, however, can save you and everyone else some time.

How much Web server do I need?

Several years ago, I went into some detail on why I think you should have your own website if you work in or want to work in technical support. Industry professionals expect you to have a website, and you can learn a lot from creating and maintaining your own.

At some point, you may decide you also want to run your own Web server. Perhaps you will opt to use an old desktop system in your old home to do the heavy lifting, or maybe you’ll sign up for a virtual private server like the ones that I use. Either way, it’s very possible that you will be running some flavor of Linux as your server’s operating system.

This leads to a natural question: how much Web server do you really need? There is an excellent chance that the answer is “a lot less than you think.” By sharing my own experiences, I hope to help you make an educated decision.

What’s your goal?

The first thing you have to nail down is exactly what your needs are. If you intend to host a ton of high-definition videos, you may need a beefy setup, but if your goal is simply to run a blog or two, run your own mail server, or set up a simple e-commerce site—or maybe even do all of the above—then you’re not likely to tax even a server with relatively low resources.

I had already been tinkering with websites for years, first on free shared hosting and then on paid shared hosting, before I took the plunge into managing my own virtual servers. I had a rough idea about what sort of traffic I’d need to be able to handle—a few thousand visitors per month—and I knew I would be hosting scans of material from my stamp collection on the oldest of my websites, Philosateleia. In addition, I wanted to start managing my own mail server for the learning experience.

What I did

In the interest of getting experience with a common server configuration, I opted to run a LAMP stack. I also decided I would like to have two separate servers: one for my websites, and one for my mail server.

I’m using Ubuntu Server, which is command line only, on a pair of virtual servers. Each has 1 GB of RAM and 20 GB of disk space as well as unlimited bandwidth, but quite frankly, unless you’re streaming video or your site becomes the next Amazon, the amount of data transfer offered with any dedicated hosting plan should be more than adequate for any traffic you’re likely to see.

My Web server is running a couple of websites plus this blog. As I mentioned earlier, we’re talking a few thousand visitors per month, which isn’t bad considering the nature of my sites. My email server with maybe a dozen or so email accounts on it is running a combination of Postfix, Dovecot, and SpamAssassin. (My email server was previously running ClamAV, which I ended up uninstalling; more on that in a bit.)

And you know what? The servers I described above are more than adequate for all of the above, and I suspect they would probably be adequate for you. Running on what are essentially bargain basement virtual private servers, I have not encountered any problems with resource demands, except…

A note on ClamAV

When I first set up my servers, ClamAV ran flawlessly, but as the years passed, I started seeing emails that had not been scanned for viruses. A bit of research and poking around in log files led me to the realization that my server didn’t have sufficient resources for ClamAV to run.

According to ClamAV’s documentation, a minimum of 1 GB of RAM is recommended in order to run it. My server just does have that, but with other packages running, it’s apparently not enough for ClamAV to run reliably.

I opted to simply uninstall ClamAV from my server. If you’re determined to have your emails scanned for viruses, I suggest going with a minimum of 2 GB of memory.

Sending Veeder-Root commands using C#

Connecting to a Veeder-Root tank level sensor unit or other automatic tank gauge unit that accepts Veeder-Root commands via telnet is a pretty simple task. I’ve discussed in the past how to do that using PuTTY, and of course there’s the good old telnet.exe included with Windows that can do the same thing.

But what do you do if you want to connect to a unit programmatically using a program written in C#?

If you’re working for an employer or customer who has tens or even hundreds of Veeder-Root units, a program of this sort may be not just handy, but necessary to speed up your day and save you from boredom.

I recently tackled a project to create a program with the sole purpose of updating the clocks on all of a customer’s Omntec units. Those units do not automatically update their internal clocks when daylight saving time begins or ends, so a unit that’s recording the correct time before daylight saving time ends up an hour behind after daylight saving time goes into effect, and so forth.

It is not my intent to reproduce my entire program here since it’s very much a niche product; however, I do want to briefly explain how to send Veeder-Root commands to a TLS unit from a C# program.

MinimalisticTelnet

What I didn’t want to do was completely reinvent the wheel when it comes to connecting to the customer’s units. Since C# telnet packages do exist, I figured one of those would probably be my best bet, and I ended up using MinimalisticTelnet, which is more than adequate for my purposes.

Connecting to an Omntec unit using the sample program provided with MinimalisticTelnet was simple enough. In the TelnetInterface.cs file, I did change the default value of TimeOutMs to 2000 instead of 100. The class is written so that the timeout is set as part of the Login function, but since the units to which I am connecting don’t require a login, that wasn’t going to work for me. (An alternative would have been to modify TelnetConnection so that I could pass in a timeout value, but I just needed something quick and dirty.

Connecting to a unit was easy enough. Sending a command and getting a response was trickier.

A word about Veeder-Root commands

When connected to a Veeder-Root or compatible TLS unit via telnet, you can issue commands ranging from setting the current time on a unit, which is what I needed to do, to querying the unit for current tank levels, and much more.

You begin each command by entering Ctrl + A. That’s easy enough on a keyboard, but how was I supposed to make it happen programmatically? The sample program provided with MinimalisticTelnet doesn’t dive into sending special keystrokes.

The solution

After quite a bit of searching, and a lot of trial and error, I stumbled across a decade-old Rebex.net blog post that mentioned using \x3 if you need to send Ctrl + C using their telnet package. Maybe \x1 would work for Ctrl + A using MinimalisticTelnet’s implementation? I gave it a try…and it worked!

For your reference, my pared down resulting code looks something like this:

TelnetConnection tc = new TelnetConnection(thisHostName, thisPort);
if (tc.IsConnected)
{
    string prompt = "\x1" + {string representing Veeder-Root command};
    tc.WriteLine(prompt);
    string response = tc.Read();
}

Hopefully this will save you some time if you’re trying to automate the process of connecting to a group of Veeder-Root, Omntec, or compatible TLS/ATG units.

How to uninstall FortiClient

A few weeks ago, I was asked to log in to one of our customers’ systems to retrieve some information for another department at my company. This particular customer uses FortiClient for VPN access; that particular tool was not installed on my laptop or my virtual machine, so I decided to download it from the FortiClient website and install it on my VM.

The installation process was straightforward, but after the software finished installing, my virtual machine slowed to an absolute crawl—100% CPU usage and all of that. My coworkers reported having encountered no problems using the same tool, so perhaps there is some other program installed on my VM that’s creating a conflict.

At any rate, after spending some time trying unsuccessfully to figure out what was causing the problem, I decided to uninstall FortiClient. I proceeded to the Programs and Features section under Control Panel in Windows, clicked on the FortiClient program listing, and…discovered that there was no uninstall button. Apps & features did have an uninstall button, but it was grayed out, so that was a no-go, too.

Thus began my multi-hour quest to try to uninstall FortiClient. If you’re looking for a quick explanation of how I uninstalled the software, jump ahead to my conclusion; otherwise, if you want a list of all the things that I tried that didn’t work, read on.

What didn’t work

I generally log in to my VM (and all of my Windows systems, for that matter) as a standard user, entering my administrator credentials whenever a UAC prompt appears. I thought maybe that was the issue, so I tried logging in to Windows using my administrator credentials instead, but encountered the same problem.

Searching on Google also turned up some suggestions that FortiClient cannot be running at the time you try to uninstall it, so I made sure to exit before opening Programs and Features and Apps & features. Again, though, I found no uninstall button.

Other comments that I read suggested that clicking the repair button, which was available in the Programs and Features window, might put FortiClient into a state where I could subsequently uninstall it. I tried that multiple times, and the uninstall button even showed up once or twice; however, when I then tried to click the uninstall button, a prompt informed me that the system had to be restarted before I could uninstall the program, and following a reboot, the uninstall button would disappear again.

Finally, I found references to a program for uninstalling FortiClient, but it is reportedly available only to FortiClient customers who have access to a restricted customer portal that requires login credentials. Since I simply downloaded the software from their website, I did not have access to that program.

What did work

After several hours of unsuccessful attempts to rid my virtual machine of the FortiClient software, I finally contacted FortiClient’s support team to explain the problems I had encountered and ask how exactly I could go about uninstalling the software.

The answer? Simply run the installer program from their website again.

I tried that, and lo and behold, the installer did indeed give me the option to uninstall. It turned out to be an incredibly easy solution, and I am very grateful that they were able to point me in the right direction, but I do wonder what the harm would be in simply having the uninstall button be visible and active on the Programs and Features and Apps & features sections in Windows.

My other takeaway is that I need to make a backup of my virtual machine files. If I run into a similar situation in the future, I should simply be able to restore those files and put my VM back in its original state rather than having to wrestle with some piece of software.

How to print 5½″×8½″ pages in booklet format using 8½″×11″ paper

Among my other activities, I’m the editor of the Local Post Collectors Society’s bimonthly journal, The Poster. In addition to writing and editing material, the job also involves layout of the journal. Until early this year, we were using an 8½″×11″ page size, but switching to a 5½″×8½″ page size seemed appealing based on the amount of content that goes into each issue.

We use Scribus for laying out The Poster, so setting the page size to what I wanted was simple enough: create a new document, change the Default Unit from points to inches, and then enter the appropriate values in the Width and Height boxes. I began setting up a test issue and soon had several pages ready to go.

Although designing using a half-page format, my goal all along was to print two pages per side of a letter-sized sheet of paper. I knew that the first and last pages would need to be on one sheet, the second and next to last pages on a second sheet, and so forth. My printer (an HP Officejet Pro 8100) has duplexing capabilities; what I didn’t realize was that I didn’t have the software to make things happen.

This is an explanation of how I finally got the half-page formatted example of The Poster ready to print. It’s entirely possible that in your role as an IT professional, someone may ask you at some point how to do something exactly this.

Scribus

My initial thought was that I should be able to do the combining of pages and preparation of the final PDF in Scribus itself. After all, it’s a desktop publishing tool, right? It should be able to handle this kind of task, right?

Wrong. Although Scribus is a really nice open-source (free) tool that can do a lot of things, it can’t combine pages onto a single sheet of paper while exporting a PDF, and it doesn’t reorder pages in preparation for duplex printing, either. I would have to find a different option.

Adobe Reader

My next thought was the print dialog in Adobe Reader. Under the “Page Sizing & Handling” section, if you click the “Multiple” button, you can specify how many pages you want on to print on each side of a sheet of paper. Okay, so maybe I would have to manually arrange my pages in Scribus so that they could print in the correct order in Adobe Reader, but being able to print multiple pages on a sheet should solve my problem, right?

Wrong again. Although I probably could have made this work on some level, Adobe Reader helpfully pads the margins of sheets printed in this manner, meaning my careful page layouts would be shrunk. I tried it out, and the result was not visually pleasing. Additional research revealed that there’s no way to change that padding behavior, so once again, I had to look for a different option.

PsUtils

I spent quite a while searching the Web, and one name kept popping up: PsUtils. Once I determined that the software already installed on my computer wasn’t going to accomplish what I wanted, I downloaded and installed PsUtils for Windows. To save myself the trouble of having to type out the full paths of the various pieces of software included in that package every time I wanted to use them, I added this to the Path environment variable in Windows:

C:\Program Files (x86)\GnuWin32\bin

I finally had the right tools, but how was I supposed to use them? That required some additional searching and experimentation, but I eventually came up with the following commands to enter at a command prompt:

  • pdf2ps "Source document.pdf" print1.ps (converts my PDF to a PS file)
  • psbook print1.ps print2.ps (reorders the pages in the PS file so that they are in the correct order for booklet printing)
  • pstops "2:0L@1(8.5in,0in)+1L@1(8.5in,5.5in)" print2.ps print.ps (the syntax will make your eyes cross, but this changes page orientation to landscape and arranges two pages on each sheet)
  • ps2pdf print.ps (converts the PS file back to PDF format)

I was nearly done at this point, but there was one final hurdle I had to clear. When I tried to print the PDF using Adobe Reader, Reader was extremely slow about sending the first page to my printer, and eventually gave up completely. I then tried opening the PDF in Google Chrome, and it printed the entire file with no complaints. Your mileage may vary, but that’s something to keep in mind.

How to use PuTTY to communicate with a TLS unit

The company that employs me is involved in the fuel management industry, helping government agencies and corporations keep track of who can dispense fuel, how much and what types of fuel they can dispense, and how much fuel has been used.

One thing that we are not directly involved in is the manufacture or maintenance of tank level sensor (TLS) units. The company’s software does include the means to track fuel levels in storage tanks, however, as reported by TLS units monitoring those tanks, and that means we need to communicate with TLS units.

Under normal circumstances, the company’s communication software once configured retrieves current tank level data from each TLS unit, but there are times when we need to independently confirm what information a unit is reporting. For example, we or a customer’s IT staff may need to determine how a contractor configured the tank probes, confirm a tank’s capacity, or manually check current tank levels or delivery records.

Our usual approach when that is necessary is to open a command prompt and start the Windows Telnet client by entering something like the following:

telnet 10.10.10.10 3001

That is assuming that the customer’s workstation or server actually has the Telnet client installed, however, which is not necessarily always the case, and we don’t always have the option of making modifications to a customer’s environment. In those situations, we have to have an alternative.

A couple of month’s ago, I started playing around with the PuTTY client to figure out if it could serve as a stand-in for the Windows Telnet client. After all, Telnet is one of the listed connection types on the main PuTTY screen, so it should be able to handle the job, right?

I began by entering a TLS unit’s IP address, changing the port number—3001 and 10001 are common defaults used by Veeder-Root and other manufacturers’ TLS units—then selected the Telnet connection type and opened the connection. I was greeted by this response:

9999FF1B

That essentially means the TLS unit looked at the information submitted to it and responded, What does that mean? I don’t know what it is you want me to do.

The solution, it turned out, was simple enough. In PuTTY, under the Connection menu, on the Telnet page, I changed the telnet negotiation mode from active to passive. After doing that and connecting to the TLS unit again, I was able to enter commands like this:

CTRL+A SHIFT+I 20100

And after pressing the enter key, I got a rational response from the TLS unit.

Curious about why I had to change this setting, I went searching for an answer, and found this in the PuTTY documentation:

“In a Telnet connection, there are two types of data passed between the client and the server: actual text, and negotiations about which Telnet extra features to use… In active mode, PuTTY starts to send negotiations as soon as the connection is opened. In passive mode, PuTTY will wait to negotiate until it sees a negotiation from the server.”

In other words, in active mode, PuTTY tries to send information about what it’s willing to do, which the TLS unit interprets as a command that it doesn’t understand. That’s why I was getting the weird response from the TLS unit I was trying to contact.

In summary, if you need to make a Telnet connection to a TLS unit using PuTTY, be sure to select passive negotiation mode.

How to fix PuTTY showing only a few files from directory

A few weeks ago, I ran into a weird issue where WinSCP would load only the /home/ directory on a Linux-based terminal, and the ls command in PuTTY would return a list of only the first few files in any directory. I’d never encountered anything like it, but since I finally figured out the problem, I thought it was well worth documenting.

The problem

It all began when one of our techs at work was installing a new terminal and needed me to connect to it to load the customer’s application. I fired up WinSCP, entered the terminal’s IP address, clicked Login, and waited. WinSCP reported that it was connecting and that it was loading the directory contents, but after a while it timed out. I tried again; same result.

Odd, I thought, running a continuous ping to check the terminal’s network connectivity. I could see a packet was getting dropped here and there, but it was nothing excessive, and definitely not something that should be preventing WinSCP from connecting. I was able to connect to the terminal via PuTTY and changed to my target directory, but when I ran the ls command, I got a list of only seven or eight files, and that was it.

I could connect to other terminals on the customer’s system without any trouble but updated WinSCP just to be sure we had the latest and greatest, but that didn’t make any difference. We chalked up the problem as a possible bad flash card and shipped out a replacement card which we tested in a terminal in our office for the tech to install. Even after he plugged in the new card, however, we continued to see the same behavior, which led me to believe that the problem was not with the terminal itself, but with the customer’s network. But what could it be?

The research

You would think that doing a search for “PuTTY and WinSCP won’t list contents of remote directory” would turn up some helpful results, but my initial efforts to figure out what was happening were fruitless. It was only after a couple of hours of searching that I finally ran across an Apple Community thread from 2011 that finally got me pointed in the right direction.

In that thread, the original poster complained about a Linux-based box “hanging” when they issued basic commands when connecting to the machine via SSH. Interesting…that sounded kind of like what I was seeing in PuTTY.

One of the users who replied, Camelot, stated that the problem was “almost certainly a MTU issue,” explaining that the original poster’s VPN was probably not resetting the MTU properly, resulting in “large packets…getting truncated and/or dropped—that’s why small transactions…work, but large ones fail.”

The solution

I’m not going to get into the weeds with a detailed explanation of MTU (maximum transmission unit), but even though no VPN was in use between the customer’s server and terminal, a MTU mismatch turned out to be exactly the problem. Somewhere on the customer’s network between the server and the terminal, there is a router or switch whether the MTU is set to less than the default packet size of 1500 used by both the customer’s Windows server and our Linux-based terminal. This was wreaking havoc with my attempts to transfer data, even basic directory listings, using WinSCP and PuTTY. This happens because the 1500-byte packet gets broken into two smaller fragments; the first makes it through, and the rest gets dumped.

Serenity Networks has a very good explanation of how to determine the MTU between two devices. It boils down to this:

  1. At a command prompt, enter ping ip -f -l 1500. If you see the message Packet needs to be fragmented but DF set, the MTU is less than 1500; proceed to step 2.
  2. Repeat the command from step 1, but reduce the last number by 10 (from 1500 to 1490), and keep doing this until you find a number that gives you normal ping results.
  3. Now, repeat the same command, but increase the last number by 1. Keep doing this until you find a number that once again gives you the Packet needs to be fragmented but DF set message; the last number that gives you normal ping results is your MTU.

In the particular environment that I was troubleshooting, MTU needed to be set to 1348. Since our terminal is Linux-based, I didn’t have to bother getting the customer’s IT team involved to try to track down the weird piece of networking equipment, instead, I was able to set the MTU on the terminal’s NIC by doing the following based on a nixCraft post:

  1. On terminal, enter sudo nano /etc/network/interfaces.
  2. Under the gateway line for eth0, enter mtu 1348, then press CTRL + X and Y to save the file.
  3. Enter sudo reboot to restart the terminal.

Once this was done, I entered ip 1 to confirm that the terminal was using the correct MTU, then fired up WinSCP. WinSCP connected immediately and displayed the remote directory on the terminal with no complaints, and I’ve had no further problems with it.

Documentation is a necessity, not a luxury

“Documentation? We don’t need no stinkin’ documentation!”

That seems to be the approach of many IT departments to keeping a record of how things work. It’s an unfortunate attitude, really. Sure, not taking the time to create documentation may save a few minutes today, but a year from now when you’re trying to recall what setting you had to change to get some piece of software to work as desired, or when you’re trying to figure out which cable end in the rat’s nest in your network closet provides connectivity to the vice president’s office, you just might find yourself wishing you had written something down.

With that in mind, I propose the following: documentation isn’t just something that’s nice to have. It’s a necessity. Skip it—or get into a job where it’s not available—and you will be pulling your hair out.

Documentation vs. no documentation

In my previous gig, we created how-to guides on how to install specialized accounting software. I worked up an “if I get hit by a bus” document explaining each of the little processes and duties I had to handle on a regular basis. We even had a map of the building with each set of network ports labeled, and on the network switches in the server room? Each of those ports was numbered to identify how it matched up with the labels on the map.

When I started my current job, however, I was in for a bit of culture shock. Documentation consisted of a couple of OneNote files, a few user manuals that had for the most part not been updated for the current versions of the various pieces of software used by the company, and a lot of, “Go ask _____, he might know how that works.”

That approach might work for a while if you’re running a one-man shop, or if your company has exceptionally low turnover, but someday someone—maybe even you—may need to know how to do something again. Why make life more difficult for your successor—or yourself—by not creating some documentation?

Thankfully, things have improved some where I work. There’s always room for further growth, of course, but the tech support group at least has a Wiki now, and I’ve been adding things to it no matter how mundane or commonplace they seem. It may not be something I need tomorrow, but a year from now, or when someone new starts, I would much rather have spent five minutes writing a how-to than have to figure it all out from scratch. And as the new guy, I would have loved to have had that kind of resource available to me.

What you can do

Chances are that you too have run into a lack of documentation. That’s not something that can be fixed overnight, especially if it has been neglected for a long time, but you can start fixing the problem now. Here are a couple of suggestions to make life easier for future you and those who follow:

  1. If your employer has an internal knowledge base of some sort, ask if you can add material. Even if it hasn’t been updated in the last five years, start squirreling away information. Post existing manuals that aren’t already there. Ask your colleagues for suggestions of things to add; even if they’re not willing to do any writing themselves, maybe they’ll at least give you some ideas of things that would be useful.
  2. Create your own knowledge base, especially when you discover things that aren’t specific to a particular workplace. That could be a blog like this one, or a Wiki, or even a bunch of Word documents stored in a folder on your computer. The exact structure is less important than making sure that information is stored somewhere that you can find it. Oh, and make sure you back it up, too. All the documentation in the world will do you no good if your hard drive goes belly up.

Creating documentation may seem like a waste of time, but I speak from experience when I say it’s worth it. Do it. Your future self will thank you, and anyone who follows in your footsteps will thank you as well.