Shared libraries on the Palm Pilot After much playing around with the Palm Pilot simulator and debugger, I have figured out how to use PalmOS's shared library features. Note that this is not "official" in any way, but it seems to work for me. Comments are welcome. This document is divided into four sections: * Intro to shared libraries on the Palm Pilot * How to use shared libraries * How to make your own shared libraries * Conclusions Note that I have only experimented with the Palm Pilot Pro; I have not tried the Palm Pilot Personal, and the old Pilots with PalmOS 1 almost certainly won't work (they don't have the SysLibLoad function, but I guess someone could emulate it...). Intro to shared libraries on the Palm Pilot Shared libraries (sometimes called "system libraries") on the Palm Pilot enable many applications to share common code, without having to have a copy of the code in each application's code resource. For example, there can be one copy of encryption routines, or a version of the standard C library, and many applications can be using it. Shared libraries can also help you get around the 32K code size limit; you can break up your code into a main portion (of at most 32K), and a number of libraries (each of at most 32K). Shared libraries are implemented as resource databases, with a resource type of libr and a resource ID of 0. In contrast, an application is a resource database with (usually) four resources: code 0, code 1, data 0, and pref 0. The libr 0 resource in a shared library corresponds exactly to the code 1 resource in an application. A shared library on the Palm Pilot is not like one on most varieties of Unix. Whereas for Unix, shared libraries export a symbol table, and the operating system's dynamic loader maps calls based on their symbol name, for the Palm Pilot, shared libraries export a dispatch table. A dispatch table is basically a list of addresses of the functions in the library that can be called from other applications. Every shared library must include the following four functions as the first four entries in its dispatch table: open Applications call this to initialize the library. close Applications call this to free storage allocated by the library, prior to exiting. sleep The system will call this routine for each shared library that is in use when the system is about to power down. wake The system will call this routine for each shared library that is in use when the system has just powered up. These functions do not need to have these exact names; the names used internally by the shared library have no relation to the names applications use to access the functions. For example, the open routine in the Serial Library is called SerOpen, and the one in the Network Library is called NetLibOpen. The remainder of the functions in the dispatch table can do whatever the library designer wants. I'm not sure what the size limit of the dispatch table is, but I haven't hit it yet (it's at most about 5500 functions, but it could be less). How to use shared libraries The Palm Pilot Pro, as shipped, comes with two libraries pre-installed: the serial library and the networking library. This section explains how to install third-party shared libraries, and how to write programs that use them. To install a shared library, simply HotSync (or use something like pilot-xfer) the .prc file to your Pilot. In my experience, if you already have a copy (or an older version) of a library on your Palm Pilot, you need to delete it (with the Memory application) before installing the new version. I'm not sure why that is, and I'll look into it more later. Once the library is installed, applications can use it. In order for a shared library to be used, it needs to be entered into the loaded library table. Once that has been done, its index in the loaded library table is called its reference number. An application that wishes to use a shared library should use the SysLibFind function. This function looks up a shared library in the loaded library table, and gives back the reference number. If SysLibFind fails, it means the library is not currently loaded. The application can try to load the library itself by using the SysLibLoad function. Once a reference number for the library has been obtained, the library's open function should be called. These functions can often be called from the StartApplication routine. In the example below, the application is trying to use the library called "Gauss Library", which is stored in a resource database of type 'libr' and creator 'Gaus': Err err; UInt GaussLibRefNum; ... err = SysLibFind("Gauss Library", &GaussLibRefNum); if (err) err = SysLibLoad('libr', 'Gaus', &GaussLibRefNum); ErrFatalDisplayIf(err, "Cannot load Gauss Library!"); err = GaussLibOpen(GaussLibRefNum); /* Check for errors in the Open() routine */ Note that SysLibLoad is not defined in some header file distributions. If you get an error regarding this, put the following near the top of your C file: Err SysLibLoad(ULong type, ULong creator, UIntPtr refNumPtr) SYS_TRAP(684); Every function in a shared library must take a UInt refNum as its first argument. For example, if the above Gauss Library implements arithmetic on Gaussian (complex) integers, and it contains a function Err GaussLibAdd(UInt refNum, Gauss_t *sum, Gauss_t *a, Gauss_t *b), the application would call it as follows: Gauss_t a, b, sum; Err err; ... err = GaussLibAdd(GaussLibRefNum, &sum, &a, &b); The application would also have #include "GaussLib.h" in it (this header file would generally be distributed with the library). The GaussLib.h header file would have definitions for any data types used by the library, as well as declarations for the library functions. For example, the declarations in GaussLib.h might be: Err GaussLibOpen(UInt refNum) SYS_TRAP(sysLibOpen); Err GaussLibClose(UInt refNum, UIntPtr numappsP) SYS_TRAP(sysLibClose); Err GaussLibSleep(UInt refNum) SYS_TRAP(sysLibSleep); Err GaussLibWake(UInt refNum) SYS_TRAP(sysLibWake); Err GaussLibCreate(UInt refNum, Gauss_t *val, Long re, Long im) SYS_TRAP(sysLibCustom); Err GaussLibRead(UInt refNum, Gauss_t *val, Long *re, Long *im) SYS_TRAP(sysLibCustom+1); Err GaussLibAdd(UInt refNum, Gauss_t *sum, Gauss_t *a, Gauss_t *b) SYS_TRAP(sysLibCustom+2); Err GaussLibMul(UInt refNum, Gauss_t *prod, Gauss_t *a, Gauss_t *b) SYS_TRAP(sysLibCustom+3); These declarations would enable applications to call, in addition to the four "standard" functions, the functions GaussLibCreate, GaussLibRead, GaussLibAdd, and GaussLibMul. Finally, when the application is about to exit, it should close the shared library by calling its close function. This is often done in the StopApplication routine. In addition, an application may choose to unload the library from the loaded library table. It should only do this, however, if the library is not in use by other applications (the close() function in the library could return information about whether any other applications are using the library). Libraries are unloaded by calling the SysLibRemove function (after which the reference number is invalid). For example, UInt numapps; err = GaussLibClose(GaussLibRefNum, &numapps); /* Check for errors in the Close() routine */ if (numapps == 0) { SysLibRemove(GaussLibRefNum); } How to make your own shared libraries This section will outline how to create a shared library using the Unix tools. A shared library is created in the same way as a regular Palm Pilot application, with a couple of notable exceptions: * The library can have no data or bss segment. This means no global or static variables. Static constants (including literal strings) are fine, though, as these go in the text (code) segment. * The m68k executable is linked with the -shared flag to gcc, which prevents the C run-time (crt0) and standard C libraries from being automatically included. Instead of the C run-time, the first object file in the link command must be the one that begins with your library's dispatch table. More on this later. We will now see how to create the Gauss Library, which was described above. Here is the complete GaussLib.h: #ifndef __GAUSSLIB_H__ #define __GAUSSLIB_H__ /* Data structures */ typedef struct { Long re, im; } Gauss_t; /* Function declarations */ Err GaussLibOpen(UInt refNum) SYS_TRAP(sysLibTrapOpen); Err GaussLibClose(UInt refNum, UIntPtr numappsP) SYS_TRAP(sysLibTrapClose); Err GaussLibSleep(UInt refNum) SYS_TRAP(sysLibTrapSleep); Err GaussLibWake(UInt refNum) SYS_TRAP(sysLibTrapWake); Err GaussLibCreate(UInt refNum, Gauss_t *val, Long re, Long im) SYS_TRAP(sysLibTrapCustom); Err GaussLibRead(UInt refNum, Gauss_t *val, Long *re, Long *im) SYS_TRAP(sysLibTrapCustom+1); Err GaussLibAdd(UInt refNum, Gauss_t *sum, Gauss_t *a, Gauss_t *b) SYS_TRAP(sysLibTrapCustom+2); Err GaussLibMul(UInt refNum, Gauss_t *prod, Gauss_t *a, Gauss_t *b) SYS_TRAP(sysLibTrapCustom+3); #endif The format of this header file is simple; the first part defines data structures, and the second declares the library functions. Note that each library function declaration is followed by a SYS_TRAP attribute. The four standard functions should have the values listed above, and subsequent functions should have arguments to SYS_TRAP starting at sysLibTrapCustom and incrementing from there. We now start looking at GaussLib.c. The first section of the C file sets up declarations for functions that the dispatch table will use, as well as defining the globals structure for the library. Remember that since the library can have no global variables, it needs to dynamically allocate space for its globals. It does this by having a single structure which contains all of its globals, and it allocates memory for this structure in the Open function, and frees it in the Close function. The only global variable we will use in the Gauss Library is a reference count of the number of applications currently using the library. The beginning of GaussLib.c, then, looks like: #include #include "GaussLib.h" /* Dispatch table declarations */ ULong install_dispatcher(UInt,SysLibTblEntryPtr); Ptr *gettable(void); /* The globals structure for this library */ typedef struct { UInt refcount; } GaussLib_globals; The first piece of code in GaussLib.c must be the code which sets up the dispatch table. For any library, it looks like this: ULong start (UInt refnum, SysLibTblEntryPtr entryP) { ULong ret; asm(" movel %%fp@(10),%%sp@- movew %%fp@(8),%%sp@- jsr install_dispatcher(%%pc) movel %%d0,%0 jmp out(%%pc) gettable: lea jmptable(%%pc),%%a0 rts jmptable: dc.w 50 dc.w 18 dc.w 22 dc.w 26 dc.w 30 dc.w 34 dc.w 38 dc.w 42 dc.w 46 jmp gausslib_open(%%pc) jmp gausslib_close(%%pc) jmp gausslib_sleep(%%pc) jmp gausslib_wake(%%pc) jmp gausslib_create(%%pc) jmp gausslib_read(%%pc) jmp gausslib_add(%%pc) jmp gausslib_mul(%%pc) .asciz \"Gauss Library\" .even out: " : "=r" (ret) :); return ret; } ULong install_dispatcher(UInt refnum, SysLibTblEntryPtr entryP) { Ptr *table = gettable(); entryP->dispatchTblP = table; entryP->globalsP = NULL; return 0; } The only part of the above code that will change for different libraries is the section between the jmptable: and out: labels. Here's how to work it out: suppose your library has n functions, including the four standard ones (for GaussLib, n is 8). Then on the first line, you would put dc.w 6n+2 (here, 6n+2 = 50). The next n lines are of the form dc.w 2n+4i-2, as i ranges from 1 to n. The next n lines are of the form jmp internal_function_name(%%pc), where each internal_function_name is the name of one of the library functions, as named in the source code for the library, which may be different from the name in GaussLib.h. (In fact, if it has the same name, you may run into problems unless you change the definition of SYS_TRAP.) The first four should be the standard four functions, and the remaining ones must be in the same order as they are listed in the header file. The last line contains the textual name of the library, which applications will use in a SysLibFind call. Now you should define the four standard functions. The open function should, if necessary, allocate memory for the library globals. Note that the amount of dynamic memory available is limited. If you have a large amount of constant data (such as a word list), include it as static const data in your program, so that it ends up in the text segment. If you really need a large amount of variable data, consider putting it into a database, and only including a pointer (or handle) to the database in the globals structure. Here is a template for the open routine: Err gausslib_open(UInt refnum) { /* Get the current globals value */ SysLibTblEntryPtr entryP = SysLibTblEntry(refnum); GaussLib_globals *gl = (GaussLib_globals*)entryP->globalsP; if (gl) { /* We are already open in some other application; just increment the refcount */ gl->refcount++; return 0; } /* We need to allocate space for the globals */ entryP->globalsP = MemPtrNew(sizeof(GaussLib_globals)); MemPtrSetOwner(entryP->globalsP, 0); gl = (GaussLib_globals*)entryP->globalsP; /* Initialize the globals */ gl->refcount = 1; return 0; } The first two lines of code are the idiom that any library function would use if it needed to access some value in the globals structure. The next section deals with the case that some other application has already opened this library. In our case, we just increment the reference count, but other libraries may want to return an error code. The section after that is the idiom for allocating the memory for a globals structure, and the last section is the library-specific initialization of the global variables. The close routine should free any memory or close any databases allocated or opened by the open routine, but only if no other application currently has the library open. Here is a template for the close routine: Err gausslib_close(UInt refnum, UIntPtr numappsP) { /* Get the current globals value */ SysLibTblEntryPtr entryP = SysLibTblEntry(refnum); GaussLib_globals *gl = (GaussLib_globals*)entryP->globalsP; if (!gl) { /* We're not open! */ return 1; } /* Clean up */ *numappsP = --gl->refount; if (*numappsP == 0) { MemChunkFree(entryP->globalsP); entryP->globalsP = NULL; } return 0; } This routine first gets a pointer to the library's global data. If this is NULL, the library is not open. This error can be signalled to the application with a result code, as above, or the library can do something like ErrFatalDisplayIf(!gl, "Gauss Library is not open!"); instead. The remainder of this routine should be used to deallocate the library global memory, if the library reference count has dropped to 0. In this case (which is a good idea), we also return the number of applications that still have the library open. If this is 0, the application that just closed us should remove the library from the loaded library table. The sleep and wake routines are usually trivial; only if your library actually cares (as, say, the Network Library might) when the Palm Pilot is turned on or off should anything happen here. Err gausslib_sleep(UInt refnum) { return 0; } Err gausslib_wake(UInt refnum) { return 0; } From here on in, you just write the code to implement your library functions. In our example, none of our functions access the library globals, so it is very easy: Err gausslib_create(UInt refNum, Gauss_t *val, Long re, Long im) { val->re = re; val->im = im; return 0; } Err gausslib_read(UInt refNum, Gauss_t *val, Long *re, Long *im) { *re = val->re; *im = val->im; return 0; } Err gausslib_add(UInt refNum, Gauss_t *sum, Gauss_t *a, Gauss_t *b) { sum->re = a->re + b->re; sum->im = a->im + b->im; return 0; } Err gausslib_mul(UInt refNum, Gauss_t *prod, Gauss_t *a, Gauss_t *b) { prod->re = (a->re*b->re)-(a->im*b->im); prod->im = (a->re*b->im)+(a->im*b->re); return 0; } We now try to compile GaussLib.c into a shared library resource database. The first step is to compile each C file into an object file. In our case, we only have one C file: m68k-palmos-coff-gcc -O5 -c GaussLib.c We now link all of our object files into a program file. If you have more than one object file, the one that starts with the dispatch table must come first. Some libraries may need the -lgcc at the end of this line, and some may not; try it without, and if you get errors, try it with. m68k-palmos-coff-gcc -shared -o GaussLib GaussLib.o -lgcc Now, make sure your data and bss segments are empty: m68k-palmos-coff-objdump --section-headers GaussLib GaussLib: file format coff-m68k Sections: Idx Name Size VMA LMA File off Algn 0 .text 00000224 00010000 00010000 00002000 2**2 CONTENTS, ALLOC, LOAD, CODE 1 .data 00000000 00000000 00000000 00000000 2**2 ALLOC, LOAD, DATA 2 .bss 00000000 00000000 00000000 00000000 2**2 ALLOC If the "Size" field of the data or bss if not 0, check your library carefully for global or static data, and either make them static constants (if possible), or else move them to the library globals structure. Next, extract the text segment from the program file, delete the data and other non-useful segments, and rename the text segment to the library segment: m68k-palmos-coff-obj-res GaussLib rm -f {code,pref,data}0000.GaussLib.grc mv code0001.GaussLib.grc libr0000.GaussLib.grc Now build a resource database (prc) file, containing only the library segment: build-prc GaussLib.prc GaussLib Gaus libr0000.GaussLib.grc At this point, everything is ready to go, except for the fact that build-prc assumes the database it creates is of type 'appl'. Shared libraries should be of type 'libr'. It would be good if a future version of build-prc would have an option for this, but for now, here is appl2libr.c: #include int main(int argc, char **argv) { FILE *prc; char kind[5]; if (argc < 2) { fprintf(stderr, "Usage: %s file.prc\n", argv[0]); exit(1); } prc = fopen(argv[1], "r+"); if (!prc) { perror("fopen"); exit(2); } if (fseek(prc, 0x3c, SEEK_SET)) { perror("fseek"); exit(2); } kind[4] = '\0'; if (fread(kind, 4, 1, prc) != 1) { perror("fread"); exit(2); } if (strcmp(kind, "appl")) { fprintf(stderr, "%s is not an appl file\n", argv[1]); exit(3); } if (fseek(prc, 0x3c, SEEK_SET)) { perror("fseek"); exit(2); } strcpy(kind, "libr"); if (fwrite(kind, 4, 1, prc) != 1) { perror("fwrite"); exit(2); } if (fclose(prc)) { perror("fclose"); exit(2); } return 0; } Compile this program, and convert your prc file to have type 'libr': gcc -o appl2libr appl2libr.c ./appl2libr GaussLib.prc You're done! GaussLib.prc is now a shared library, ready to be uploaded to a Palm Pilot. You can distribute the prc file along with applications that use it, and/or you can distribute the header file so people can write their own applications that use the library. For reference, here is a Makefile: LIBNAME = GaussLib CREATOR = Gaus CC = m68k-palmos-coff-gcc XTRALIBS = -lgcc OBJRES = m68k-palmos-coff-obj-res BUILDPRC = build-prc $(LIBNAME).prc: libr0000.$(LIBNAME).grc ./appl2libr $(BUILDPRC) $@ $(LIBNAME) $(CREATOR) libr0000.$(LIBNAME).grc ./appl2libr $@ ./appl2libr: appl2libr.c gcc -o $@ $< libr0000.$(LIBNAME).grc: $(LIBNAME) $(OBJRES) $(LIBNAME) rm -f {code,pref,data}0000.$(LIBNAME).grc mv code0001.$(LIBNAME).grc libr0000.$(LIBNAME).grc $(LIBNAME): $(LIBNAME).o $(CC) -shared -o $(LIBNAME) $(LIBNAME).o $(XTRALIBS) $(LIBNAME).o: $(LIBNAME).c $(CC) -O5 -c $(LIBNAME).c clean: rm -f libr0000.$(LIBNAME).grc $(LIBNAME).o spotless: clean rm -f appl2libr $(LIBNAME).prc $(LIBNAME) Conclusions Success and failure reports, comments, and questions are welcome. Depending on the content, appropriate fora are the pilot.programmer and pilot.programmer.gcc newsgroups hosted on news.massena.com, and the pilot-unix mailing list at . If necessary, I can be reached directly at the address below (be warned that my email queue sometimes gets quite backlogged). Back to the ISAAC Group's Pilot page ------------------------------------------------------------------------ IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS. Ian Goldberg, iang@cs.berkeley.edu