https://wiki.preterhuman.net/index.php?title=Whattohide.shar&feed=atom&action=historyWhattohide.shar - Revision history2024-03-28T08:05:12ZRevision history for this page on the wikiMediaWiki 1.35.0https://wiki.preterhuman.net/index.php?title=Whattohide.shar&diff=16763&oldid=prevNetfreak: Created page with "<pre> #! /bin/sh # This is a shell archive, meaning: # 1. Remove everything above the #! /bin/sh line. # 2. Save the resulting text in a file. # 3. Execute the file with /bin/..."2020-09-01T06:08:08Z<p>Created page with "<pre> #! /bin/sh # This is a shell archive, meaning: # 1. Remove everything above the #! /bin/sh line. # 2. Save the resulting text in a file. # 3. Execute the file with /bin/..."</p>
<p><b>New page</b></p><div><pre><br />
#! /bin/sh<br />
# This is a shell archive, meaning:<br />
# 1. Remove everything above the #! /bin/sh line.<br />
# 2. Save the resulting text in a file.<br />
# 3. Execute the file with /bin/sh (not csh) to create the files:<br />
# README<br />
# update-all-to-hidden<br />
# whattohide<br />
# update-to-hidden<br />
# ChangeLog<br />
# This archive created: Fri Jun 30 13:37:32 2000<br />
export PATH; PATH=/bin:$PATH<br />
if test -f 'README'<br />
then<br />
echo shar: will not over-write existing file "'README'"<br />
else<br />
sed 's/^X//' << \SHAR_EOF > 'README'<br />
X<br />
XThese comments and the scripts and utility<br />
Xprogram mentioned are freely available.<br />
X<br />
XThey are useful to folks who build applications<br />
Xwith large numbers of DSOs. <br />
XNothing here can or should be used to affect<br />
Xsystem DSOs. Only to change the build<br />
Xof DSOs you build for your application.<br />
XIf the number of DSOs you build for your application is<br />
Xgetting to be more than 10 you might consider using<br />
Xthe facilities here.<br />
XIf the number of DSOs you build for your application is<br />
Xnear or above 100 you should definitely use the following.<br />
X<br />
XReport problems or suggestions to me, davea@sgi.com.<br />
X<br />
X<br />
X<br />
Xdavea@sgi.com Feb 2000<br />
X$Revision: 1.2 $ <br />
X$Date: 2000/06/30 20:36:32 $<br />
X=========================<br />
XDSO: a shared library. DSO is the<br />
Xname used in IRIX documentation. <br />
XDSO is shorter to type :-)<br />
X<br />
X*****The Problem<br />
X(back of the envelope style calculation, based<br />
Xon knowledge of run time lookup internals):<br />
XLookup N syms by name with 1 DSO to search: 2N<br />
XLookup N syms by name in N DSOs by name,<br />
Xfinding 1/2 way thru list on average: (2N/2)*2N<br />
XQuadratic cost! Bad!<br />
X<br />
X*****The Summary <br />
XIf one has more than a few DSOs<br />
Xlookup can become expensive.<br />
XThis can cause application startup time to be excessive and<br />
Xnoticeable when the numbers get big enough.<br />
XFor 70000 symbols and 100 DSOs the time gets to <br />
Xseveral seconds.<br />
XEven for a single DSO, if the lookups are<br />
Xactually not necessary, there is time wasted.<br />
X<br />
X*****Justifying the Problem Calculation:<br />
XIterating thru the list of DSOs takes cycles.<br />
XFinding if a symbol is in a given DSO is fast (since<br />
Xthe Elf hash table is effective, it is an O(1)<br />
Xoperation) but lets call that<br />
Xthe same cost as getting to a single DSO in the list.<br />
XSo each by-name lookup in a single DSO<br />
Xhas cost 1 to find the DSO, and 1 to find the name.<br />
XN names to look up in 1 DSO, cost is thus 2N.<br />
X<br />
X*****A Possible Solution:<br />
XMany of the names so lookup up are probably resolved<br />
Xin the *same* dso that the call is in.<br />
XSuch names can be converted to be 'hidden' symbols<br />
Xand hidden symbols are *never* looked up by name.<br />
XEvery name hidden gets out of the quadratic time bind!<br />
X<br />
XBuild the DSOs and app (no changes from present build)<br />
XCreate a list of the DSOs and the executable for input<br />
Xto a script.<br />
Xls -1 is the best approach: you want the <br />
Xexecutable and your DSOs, not<br />
Xsystem DSOs, on the list.<br />
X<br />
XCreate the list of hideable symbols (script name:whattohide)<br />
X<br />
X******Hide for testing purposes<br />
XNow use the list of DSOs and the list of hideable symbols<br />
Xto hide the hideable symbols (scripts update-all-to-hidden <br />
Xand update-to-hidden and the program 'optionalsym').<br />
X(don't bother hiding symbols in the executable).<br />
X<br />
XThere is no formal guarantee that using 'optionalsym'<br />
Xproduces a valid DSO (in fact in once sense the result<br />
Xis incorrect). However, all existing versions of<br />
Xrld(1) account for this incorrectness and the<br />
Xresult should work.<br />
X<br />
X<br />
X******Hide for shipment of your application<br />
XFor build of a real product, the use the ld option<br />
X-hiddens_file <filename><br />
Xwhere the <filename> contains the list of symbols to be hidden.<br />
XThis generates a DSO with the hidden symbols etc<br />
Xproperly laid out so the DSO is correct (unlike optionalsym)<br />
X<br />
X<br />
XIf the total list of filenames too large, the -hiddens_file <filename><br />
Xcould expand ld to the point it runs out of virtual memory.<br />
XIn this case, do something like the following<br />
X nm -Bg <DSONAME> |grep ' T ' | awk ' { print $3 } ' >temp-sto1<br />
X cat temp-sto1 <TOHIDELIST> |sort |uniq -d >temp-sto2<br />
Xborrowed from the update-to-hidden script<br />
Xand use "-hiddens_file temp-sto2", which has just the names<br />
Xthat this DSO should hide.<br />
X<br />
X<br />
X*****When you have functions that must not be hidden<br />
XSome applications allow users to add new DSOs that refer<br />
Xback to the base DSOs or executable. In that case, the<br />
Xlist of 'symbols to hide' generated by the scripts<br />
Xwill have some things that the script cannot know must<br />
Xbe left visible. <br />
X<br />
XThe solution: simply add the required stuff to the set <br />
Xof symbols referred to externally (modify the whattohide<br />
Xscript to add in those symbols as extra outside-dso references).<br />
X<br />
XNow this solution is simple for C (and reasonably so for Fortran)<br />
Xbut since, for C++, one might want an entire class's<br />
Xfunctions to be visible, C++ is harder.<br />
XYou may need to compile certain headers and use the demangler<br />
X(/usr/lib/c++/c++filt) to allow you to match up<br />
Xnames-to-be-left-visble by the class name.<br />
XAt this time no code for such is provided here.<br />
XThere is no generally agreed upon way to mark classes as<br />
Xto-be-exported so it's not clear there is a single<br />
Xapproach.<br />
X<br />
X<br />
X*****General comments<br />
XIf you leave out a DSO or the executable in<br />
Xthe process of building the initial DSO list<br />
Xthen when all is done the app may not work (rld(1)<br />
Xmay say it cannot find symbols at app run time)!<br />
XIf you miss an executable or a DSO in the<br />
Xhiding the result will still work but<br />
Xsome of the savings will be missed.<br />
X<br />
X*****Testing for correct execution:<br />
XIt's always wise to test your app with LD_BIND_NOW <br />
Xset to 1, as that is the quickest way to verify that<br />
Xthere are no name resolution surprises.<br />
XAs opposed to having a user reporting an app crash<br />
Xbecause rld could not resolve a symbol due to using<br />
Xa code path that did a call not used in testing.<br />
XLD_BIND_NOW ensures all the calls actually have code<br />
Xsomewhere to resolve them! <br />
XThis still requires full testing though, as you must<br />
Xbe sure all dlopens are done...<br />
X<br />
X<br />
SHAR_EOF<br />
fi # end of overwriting check<br />
if test -f 'update-all-to-hidden'<br />
then<br />
echo shar: will not over-write existing file "'update-all-to-hidden'"<br />
else<br />
sed 's/^X//' << \SHAR_EOF > 'update-all-to-hidden'<br />
X#!/bin/ksh<br />
X#<br />
X# Usage: update-all-to-hidden <filename> <names to hide><br />
X# where <filename> contains a list of DSOs, one per line,<br />
X# full or relative path, that you want some symbols hidden in.<br />
X# where <names to hide> contains a list of names to hide,<br />
X# one per line, across all the DSOs.<br />
X# The script whattohide produces such a list.<br />
X#<br />
X# $Revision: 1.3 $<br />
X# $Date: 2000/06/30 20:37:00 $<br />
X#set -x<br />
Xif [ $# != 2 ]<br />
Xthen<br />
X echo "Usage: update-all-to-hidden <file-with-dsos> <hide-list>"<br />
X exit 1<br />
Xfi<br />
X<br />
Xwhile read dsoname<br />
Xdo<br />
X ksh update-to-hidden $dsoname $2<br />
Xdone <$1<br />
SHAR_EOF<br />
fi # end of overwriting check<br />
if test -f 'whattohide'<br />
then<br />
echo shar: will not over-write existing file "'whattohide'"<br />
else<br />
sed 's/^X//' << \SHAR_EOF > 'whattohide'<br />
X#!/bin/ksh<br />
X# $Revision: 1.6 $<br />
X# $Date: 2000/02/16 18:15:20 $<br />
Xif [ $# != 1 ]<br />
Xthen<br />
X echo "Usage: whattohide file-with-dsos-listed"<br />
X exit 1<br />
Xfi<br />
X<br />
Xo=hides-list<br />
Xt=temp-hides<br />
X# This script takes as input a file with<br />
X# a list of pathnames to the DSOs in an application.<br />
X# Plus the executable name.<br />
X# It produces a list of names which can be 'hidden'<br />
X# and can thus get us out of the time bind.<br />
X# If the list of names is substantial using the<br />
X# list is worthwhile.<br />
X#<br />
X# System DSOs can be left out of the list (but it's not<br />
X# harmful to leave them in).<br />
X# Leaving out DSOs that are part of the app<br />
X# will most likely break the application!<br />
X#<br />
X# If you miss an executable or a DSO in the<br />
X# rebuild the result will still work but<br />
X# some of the savings will be missed.<br />
X#<br />
X# This script creates temp files in the local directory.<br />
X#<br />
X<br />
X#==========================================<br />
Xl=$1<br />
X#set -x<br />
Xrm -f $t-allsyms<br />
Xrm -f $t-U<br />
Xrm -f $t-T<br />
Xrm -f $t-defwhere<br />
Xrm -f $t-symdetails<br />
Xrm -f $t-wherelist<br />
Xrm -f $t-all<br />
Xrm -f $t-hide<br />
Xrm -f $t-alreadyhidden<br />
Xrm -f $t-hid<br />
Xrm -f $t-prot<br />
Xrm -f $t-dup<br />
X<br />
X# Find all the global symbols<br />
Xwhile read file<br />
Xdo<br />
X nm -Bg $file >>$t-allsyms<br />
Xdone < $l<br />
X<br />
X# Create a list of undefined symbols<br />
Xgrep ' U ' $t-allsyms >$t-U<br />
X# Create a list of defined text symbols (which could<br />
X# be hidden or protected.... check below)<br />
Xgrep ' T ' $t-allsyms >$t-T<br />
X<br />
X# We only deal with text as only those can we be<br />
X# quite sure it is ok to simply hide.<br />
X# Common and Data are problematic.<br />
X<br />
X# The number of dups is interesting. Might<br />
X# suggest a build problem?<br />
Xcat $t-T | awk ' { print $3 }' |sort |uniq -d >$t-dup<br />
Xecho "duplicate names file entry count" `wc -l $t-dup`<br />
X<br />
X# We want to remove things whch are already<br />
X# marked as 'HIDDEN' from our hides list. elfdump -Dt tells us this<br />
X# (once we process it later)<br />
X<br />
X<br />
X<br />
X# effectively sorts U (undef) before T (definition, text)<br />
Xcat $t-U $t-T |uniq >$t-all<br />
X<br />
X# Emit only those not marked as undefined<br />
X#<br />
Xcat >$t.1.awk <<\_EOF<br />
X/ U / { undef[$3] = 1 }<br />
X/ T / { if ( undef[$3] != 1 ) {print $3 ; } }<br />
X/ D / { if ( undef[$3] != 1 ) {print $3 ; } }<br />
X/ A / { if ( undef[$3] != 1 ) {print $3 ; } }<br />
X_EOF<br />
Xawk -f $t.1.awk <$t-all >$t-hide<br />
X<br />
X<br />
X# Emits SVR4 style name so we can see HIDDEN PROTECTED<br />
Xwhile read file<br />
Xdo<br />
X elfdump -Dt $file >>$t-symdetails<br />
Xdone < $l<br />
X<br />
X# Things that are already hidden should not<br />
X# be in the list! make lists of these<br />
Xgrep -e '.*[ ]*HIDDEN[ ]*' <$t-symdetails >$t-hid<br />
Xgrep -e '.*[ ]*PROTECT[ ]*' <$t-symdetails >$t-prot<br />
X<br />
X# Create a new file with things we want to hide and protect first<br />
X# Do this in specific order, effectively sorting by<br />
X# category.<br />
Xawk ' {print "HIDDEN " $8 }' <$t-hid >$t-thc<br />
Xawk ' {print "PROTECT " $8 }' <$t-prot >>$t-thc<br />
Xawk ' {print "CHECK " $1 }' <$t-hide >>$t-thc<br />
X<br />
X<br />
X# Emit as hideable only those not hidden or protected<br />
X# Removes hidden and protected symbols.<br />
X# Depends on input ordering to work correctly.<br />
Xcat >$t.2.awk <<\_EOF<br />
X/^HIDDEN / { hid[$2] = 1; }<br />
X/^PROTECT / { hid[$2] = 1; }<br />
X/^CHECK / { if ( hid[$2] != 1 ) { print $2; } }<br />
X_EOF<br />
Xawk -f $t.2.awk <$t-thc > $o<br />
X<br />
X<br />
Xecho "number of externs that should be hidden " `wc -l $o`<br />
X<br />
Xexit 0<br />
SHAR_EOF<br />
fi # end of overwriting check<br />
if test -f 'update-to-hidden'<br />
then<br />
echo shar: will not over-write existing file "'update-to-hidden'"<br />
else<br />
sed 's/^X//' << \SHAR_EOF > 'update-to-hidden'<br />
X#!/bin/ksh<br />
X#<br />
X# Given a DSO name, and the file hides-list with names to hide,<br />
X# change those names to STO_HIDDEN<br />
X#<br />
X# Usage: update-to-hidden <DSOname> <hideslist name><br />
X#<br />
X# $Revision: 1.3 $<br />
X# $Date: 2000/06/30 20:37:00 $<br />
X#set -x<br />
X<br />
Xif [ $# != 2 ]<br />
Xthen<br />
X echo "Usage: update-to-hidden <dso pathname> <hide-list>"<br />
X exit 1<br />
Xfi<br />
X<br />
Xn=$1<br />
Xh=$2<br />
X<br />
Xrm -f temp-sto1<br />
Xrm -f temp-sto2<br />
X<br />
Xnm -Bg $n |grep ' T ' | awk ' { print $3 } ' >temp-sto1<br />
X<br />
Xcat temp-sto1 $h |sort |uniq -d >temp-sto2<br />
X<br />
Xwhile read name<br />
Xdo<br />
X optionalsym -u STO_HIDDEN -y $name $n<br />
Xdone <temp-sto2<br />
X<br />
X#Marked hidden now, we trust.<br />
SHAR_EOF<br />
chmod +x 'update-to-hidden'<br />
fi # end of overwriting check<br />
if test -f 'ChangeLog'<br />
then<br />
echo shar: will not over-write existing file "'ChangeLog'"<br />
else<br />
sed 's/^X//' << \SHAR_EOF > 'ChangeLog'<br />
X<br />
X30 Jun 2000 davea@sgi.com<br />
X Cheng Liao contributed fixes to typographical errors in<br />
X usage messages and an 'if'.<br />
X<br />
X I, davea, contributed some more words in the README for a<br />
X special case: externally-visible-stuff.<br />
SHAR_EOF<br />
fi # end of overwriting check<br />
# End of shell archive<br />
exit 0<br />
</pre><br />
<br />
[[Category:Source Code]][[Category:SGI]]</div>Netfreak