Installing and Accessing Shared Libraries

Following on from last week, you should have been able to build your own .so library file and link to it with a separate executable.

This week we’ll look at installing the shared library (see last week for the source files. This weeks commands follow on from there).

Basically we have three options when it comes to building and including shared libraries.

rpath

You’ve already seen one option, which is to specify the location of the library with -L and also pass an -rpath to the linker so that it embeds that path in the executable stating where the library can be found (so the loader will be able to locate it at runtime).

Using rpath is fine, but if you share your shared library with others, they all need to make sure it lives in the same directory that you have specified. In my case, the library was located in my home directory, faye, which wouldn’t be much good if I wanted anyone else to use my program on their computer!

LD_LIBRARY_PATH

This second method uses the environment variable LD_LIBRARY_PATH. You can see what it is currently set to with the command:

echo $LD_LIBRARY_PATH

This should be blank (unless you have previously used it for something!)

To use it, you just need to set it to also include the location of your shared object:

export LD_LIBRARY_PATH=/home/faye:$LD_LIBRARY_PATH

Now if we echo the command we see:

/home/faye:

And at compile time, we can create an executable that links to the library with the following line:

g++ -Wall -L/home/faye/sotest main.cpp -lpal

We pass the path to the linker with the -L option above and the LD_LIBRARY_PATH variable tells the loader where to find the library when we run the executable (so we no longer need the rpath option to embed the path into the executable).

Problems…?

LD_LIBRARY_PATH isn’t really the optimal solution – it’s good for testing because you don’t need root access to use it, but you probably don’t want to distribute code that relies on LD_LIBRARY_PATH being set.

Before we move on, clear the LD_LIBRARY_PATH variable with:

unset LD_LIBRARY_PATH

ldconfig

This is the ‘official’ way to install your shared library, and it requires root privileges.

First of all, the location we’re going to use is the standard /usr/lib.

You need to copy the library to that location (which you will need to use sudo for, or login as root to perform):

sudo mv /home/faye/libpal.so /usr/lib

Next run ldconfig, which will update the cache of libraries available in the standard system directories:

sudo ldconfig

You can check if the cache has been updated by requesting the cache and grepping it for our libpal library:

ldconfig -p | grep libpal
    libpal.so (libc6,x86-64) => /lib/libpal.so    //here it is!

Now we can compile our executable exactly as we would normally (and as we originally tried last week):

g++ -Wall main.cpp -lpal

We don’t need the rpath (the loader finds the library via the ldconfig cache), we don’t need the -L option (the linker finds the library in the standard search paths), and it runs exactly as it should.

Hooray!



Creating a Shared Library

I thought we could take a quick look at how to create a shared library out of our code. This week we’ll create the library and next week we’ll look at the various ways of installing/accessing it on the operating system.

I’m going to re-use the palindrome program I’ve talked about before (a proper version, not the dodgy one).

First of all we want to break the code up into several files instead of having it all in one cpp file. We’ll take the palindrome function out and use that to create a library. Obviously if you were creating your own libraries you’d probably want a lot more functionality, but I’ll leave that part to you 😉

Right then. Let’s create a header file, pal.h:

bool isPalindrome(char* word);

And a cpp file, pal.cpp:

#include "pal.h"
#include <string.h>

bool isPalindrome(char* word)
{
    bool ret = true;

    char *p = word;
    int len = strlen(word);
    char *q = &word[len-1];

    for (int i = 0 ; i < len ; ++i, ++p, --q)
    {
        if (*p != *q)
        {
            ret = false;
        }
    }

    return ret;
}

And then get down to the business of creating our shared object!

First of all we want to compile the code above into an object file. To do this we pass gcc the -c option, which tells it NOT to perform the linking stage (if you did try to link, gcc would complain that you have no main function defined – because a program can’t run without a main, but a library doesn’t need one).

g++ -fPIC -c -Wall pal.cpp

This will create a pal.o file in the directory that you are working in.

Next, we want to create our actual library with this line, which I’ll explain below:

ld -shared pal.o -o libpal.so

This uses the linker program (ld), usually called by g++ (remember we told g++ with the -c option not to link in the first stage). It says make a shared object (the -shared option), using the input file pal.o and call it libpal.so (the -o option). The .so extension is the usual naming convention for shared libraries.

After running this, you should be able to see the libpal.so file in your working directory.

Cool! You’ve just created a shared library :-)

Next up we actually want to use that library with some other code. So let’s create  a main.cpp file that calls the library function isPalindrome:

#include "pal.h"
#include <iostream>

using namespace std;

int main()
{
    while (1)
    {
        char buffer[64] = {0};
        cin >> buffer;

        if (isPalindrome(buffer))
        {
            cout << "Word is a palindrome" << endl;
        }
        else
        {
            cout << "Word is not a palindrome" << endl;
        }
    }

    return 0;
}

As with all libraries, we use it in main.cpp by including the library header (pal.h). Then at compile time we have to tell gcc that we want to link to the library.

We would normally link to a library with a line like this:

g++ -Wall main.cpp -lpal

However, if we try this, we get an error that says:

/usr/bin/ld: cannot find -lpal
collect2: error: ld returned 1 exit status

Oops.

This is because the linker, ld, looks in a specified set of locations for libraries and our library doesn’t live there (more on this next week).

So what can we do?

A temporary solution is to tell ld where the library currently lives. It’s in our working directory, so we can use the -rpath option to let the linker know about it.

g++ -Wall -L/home/faye -Wl,-rpath=/home/faye/ main.cpp -lpal

g++ -Wall is our usual compile command, checking for all warnings.

-L is the path to the shared library, provided so that the linker knows where to look for it.

-Wl means a list of comma separated options for the linker follows, and the only option we pass is:

-rpath – which means that the path the library will be embedded in the executable so that the loader will be able to find it at run time.

Finally we have our main.cpp file, and then we add in our new shared library by filename with libpal.so.

Easy!

Just to make that clear:

-L is used for the linker
-rpath embeds the path (via the linker) for the loader.

If you omit either of these, you will have problems either linking or running.

If you run this now, with ./a.out, you’ll get the following:

so

We’ll look at other ways of accessing shared libraries next week (i.e. without using -rpath).

In the meantime, have fun!



A Challenge Discussion – Returning Pointers To Local Variables

Last week’s challenge was a tough one, so don’t be alarmed if you had no idea what the issue might have been.

Also, it can be hard to spot problems in a functional program because you tend to concentrate on what the code is actually doing, rather than the validity of the code itself.

To give you a clue, before I talk about last week’s code, here’s a very small version with a similar problem:

#include <iostream>
#include <string.h>

using namespace std;

int* func()
{
    int num = 256;
    int* tmp;
    tmp = &num;
    return tmp;
}

int main()
{
    int* x = func();
    cout << "Value of x is: " << *x << endl;
    return 0;
}

You might spot this one immediately, but you can be forgiven for missing even this.

The output of this program is as you would expect: it prints 256 to the screen.

However, what if I modify the code to add a couple of extra (unrelated) lines:

#include <iostream>
#include <string.h>

using namespace std;

int* func()
{
int num = 256;
int* tmp;
tmp = &num;
return tmp;
}


int main()
{
int* x = func();

char tmp[] = "hello world"; //extra line of code
int len = strlen(tmp);      //extra line of code

cout << "Value of x is: " << *x << endl;
return 0;
}

Now, the output of the program (on my machine) is zero.

What!?

Yes, that’s right. Apparently, creating an array of chars and finding the length of it has messed up my original code. Surely not?

In fact, that is exactly what has happened.

If you look at func, you can see that I’m creating an int and setting it to 256. All good.

Then I create a pointer to an int and point it to 256. Still okay.

Then I return the pointer from func so someone else can see what it points to.

Uh oh!!

As soon as I exit func, the variables all go out of scope. That means that they no longer exist and whether or not they return the right value is down to pure chance.

In the first program, I get lucky, and it prints correctly, because 256 just happens to be sitting in memory undisturbed. It’s still there, but only because nothing has used that memory – which is now considered free and available to use. In the second program however, the additional lines that call strlen overwrite the stack and my original 256 value is lost forever. Bah!

As you can see, even with -Wall, there is NO compiler warning about this, and if you get lucky (as in program 1), you might never even know a bug was there. In fact, if the value you are expecting falls within a wide range, rather than being set at say 256, there is every chance that you could run this code for years and be none the wiser that the values you are getting back are in fact totally unrelated to what you are expecting (I have seen this exact bug in real life).

So, the moral of this is: never return a pointer to a local variable.

Now let’s go back to last week’s code and see what’s happening:

#include <iostream>
#include <string.h>

using namespace std;

char* isPalindrome(char* word)
{
    char* ret = 0;
    ret = (char*)"Yes! This is a palindrome.";
	
    char *p = word;
    int len = strlen(word);
    char *q = &word[len-1];

    for (int i = 0 ; i < len ; ++i, ++p, --q)
    {
        if (*p != *q)
        {
            ret = (char*)"This is not a palindrome.";
        }
    }
    return ret; 
} 

int main()
{ 	
    while (1)
    {
        char buffer[16] = {0};
        cin >> buffer;

        char* result = isPalindrome(buffer);

        cout << result << endl;
    }
    return 0;
}

In this code, we are doing a very similar thing. In the isPalindrome function, we’re declaring a pointer to a char and returning it to the main function. This is bad, right? Because once the isPalindrome function returns, all the variables will go out of scope.

Well, it turns out to be even more complex than that (I told you this was a tough one, but bear with me – it’s totally worth reading to the end!).

You would expect that even though this program runs correctly at the moment, if the program was larger and had more code, there would be every chance that something would overwrite the stack and you wouldn’t get the right result back. But the thing is, if you try to break this code you can’t. It doesn’t matter what you do, how many more variables or functions you add, how much stack space you use up, it always runs correctly.

How can this be possible?

Here’s a clue.

If you change the main function to add this line you get a segfault at run time:

int main()
{ 	
    while (1)
    {
        char buffer[16] = {0};
        cin >> buffer;

        char* result = isPalindrome(buffer);
        result[0] = 'x'; //ADD THIS LINE HERE

        cout << result << endl;
    }
    return 0;
}

Why on earth?

The program segfaults because by trying to alter the string that is returned you are actually accessing read-only data.

How come?

When the program is compiled, string literals (like “This is not a palindrome.”), are treated differently to everything else and are stored in a special area of read-only memory which is neither the heap nor the stack. In fact, if you examine the memory addresses of the variables in gdb, you would see that the address that char* ret points to is very different to the all the other addresses that are allocated to the program variables.

So, in this example, we return a pointer to an address in read-only memory and don’t have to worry about anything overwriting it, because it is safely tucked away from the rest of the code!

However, when someone tries to write to what they might assume is a character array, using the line added above, they would get a segfault.

Nasty, eh?

So, the big question, is this correct correct and valid, or not?

I’d say not really. Firstly, you shouldn’t be returning a pointer to a local variable – in this case you’d get lucky and it would always work. However, better practice would be to declare the string as const, so a programmer couldn’t try to modify it without realising the consequences. It should also be made 100% clear with a comment that this was intended and you aren’t just returning a pointer to a local variable without understanding what the consequences are generally. Finally, you should be aware that this isn’t always the case – some compilers/processors may implement string literals differently. Overall verdict? Don’t do it.

Just before I finish up, if you’re wondering how to ensure that your string literal is actually a char array that is writeable and not a string literal in read-only memory, the difference is as follows:

char *x = "This is a string literal";
char y[] = "This is a character array";

char y above is another way of writing the following:

char y[] = {'T','h','i','s',' ','i','s',' ','a',' ','c','h','a','r','a','c','t','e','r',' 'a','r','r','a','y','\0'};

Scoring

One point for recognising that I shouldn’t have returned a pointer to a local variable.

A big bonus point for recognising that I was using a string literal that couldn’t be modified even if it was successfully accessible after the function exits.

Did you learn something new?

I told you it was worth reading to the end 😉

Have a good week!



A Challenge

I have a challenge for you. Is there a problem with this code?

It compiles without warnings, and runs as expected.

I’ll reveal all next week.

Have fun!

g++ -g -Wall palindrome_problem.cpp
#include <iostream>
#include <string.h>

using namespace std;

char* isPalindrome(char* word)
{
    char* ret = 0;
    ret = (char*)"Yes! This is a palindrome.";
	
    char *p = word;
    int len = strlen(word);
    char *q = &word[len-1];

    for (int i = 0 ; i < len ; ++i, ++p, --q)
    {
        if (*p != *q)
        {
            ret = (char*)"This is not a palindrome.";
        }
    }
    return ret; 
} 

int main()
{ 	
    while (1)
    {
        char buffer[16] = {0};
        cin >> buffer;

        char* result = isPalindrome(buffer);

        cout << result << endl;
    }
    return 0;
}