Hello,
I recently wrote bindings to the C-library Shapelib (it reads/writes a common file format used in Geographic Information Systems).

I've been trying to write a small test program to make sure my bindings 'work' and I've come across a bizarre memory bug.  I  THINK I've identified the code that causes the problem, but I have no idea why.

My test-suite includes this function:

void shapeRead(string filename) {
  SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), "rb" );

  int n, shp_type;
  double pad_min_bound, pad_max_bound;

  SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, &pad_max_bound);

  SHPClose( hShp );
}

If I comment out the SHPGetInfo call, then the segmentation fault doesn't happen, but if its there then the program segfaults AFTER the shapeRead function exits (eg. SHPClose runs fine) ).

In fact if I call SHPGetInfo as follows, the crash doesn't occur:
SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL pointer last arg.

So in C the function SHPGetInfo is:

void SHPAPI_CALL
      SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * pnShapeType,
                  double * padfMinBound, double * padfMaxBound );

While my D binding is (pretty much the same):

extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities,
	  int* pnShapeType, double* padfMinBound, double* padfMaxBound );


I have no idea what is going on.  The sizes of ints and doubles are 4 and 8 bytes using gcc on my system (which is how I compiled my C library) so those match the sizes of the corresponding D types [I thought maybe there was a type-size mismatch and that was causing something to be overwritten, but it doesn't appear that way].

Any hints on where to look next would be appreciated.

Craig

My strategy here would be to:
A.  run the program in a debugger, say GDB, to get a exhaustive stacktrace for hints about where to look at.
B. have a quick look at the library directly (the "Use the Source Luke" strategy).

Since I was curious about your problem (you had everything correct - this should not fail), and have no access to your code, I checked out the shapelib code from its cvs public repository and found out that the last pointer, `double * padfMaxBound` is actually a pointer to an array of 4 elements: in shapelib/shpopen.c: SHPGetInfo: starting line 823:
    for( i = 0; i < 4; i++ )
    {
        if( padfMinBound != NULL )
            padfMinBound[i] = psSHP->adBoundsMin[i];
        if( padfMaxBound != NULL )
            padfMaxBound[i] = psSHP->adBoundsMax[i];
    }

This also explains why it does not segfault when you pass a null pointer: no array out of bounds happen then.

padfMaxBound should point to a `double []` or `double [4]` instead of a mere `double`.

I also ended checking the API documentation (http://shapelib.maptools.org/shp_api.html) and they do document that padfMinBound and padfMaxbound are pointers to 4 values X, Y, Z and M in a four entry array.
You may want to check if there isn't any other of those "C tricks" hiding in your D bindings.



On 12/07/2013 04:29 PM, Craig Dillabaugh wrote:
> Hello,
> I recently wrote bindings to the C-library Shapelib (it reads/writes a
> common file format used in Geographic Information Systems).
>
> I've been trying to write a small test program to make sure my bindings
> 'work' and I've come across a bizarre memory bug.  I THINK I've
> identified the code that causes the problem, but I have no idea why.
>
> My test-suite includes this function:
>
> void shapeRead(string filename) {
>    SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), "rb" );
>
>    int n, shp_type;
>    double pad_min_bound, pad_max_bound;
>
>    SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, &pad_max_bound);
>
>    SHPClose( hShp );
> }
>
> If I comment out the SHPGetInfo call, then the segmentation fault
> doesn't happen, but if its there then the program segfaults AFTER the
> shapeRead function exits (eg. SHPClose runs fine) ).
>
> In fact if I call SHPGetInfo as follows, the crash doesn't occur:
> SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL pointer
> last arg.
>
> So in C the function SHPGetInfo is:
>
> void SHPAPI_CALL
>        SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * pnShapeType,
>                    double * padfMinBound, double * padfMaxBound );
>
> While my D binding is (pretty much the same):
>
> extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities,
>        int* pnShapeType, double* padfMinBound, double* padfMaxBound );
>
>
> I have no idea what is going on.  The sizes of ints and doubles are 4
> and 8 bytes using gcc on my system (which is how I compiled my C
> library) so those match the sizes of the corresponding D types [I
> thought maybe there was a type-size mismatch and that was causing
> something to be overwritten, but it doesn't appear that way].
>
> Any hints on where to look next would be appreciated.
>
> Craig
>
>
>

On 12/07/2013 03:11 PM, Rémy Mouëza wrote:

> the last pointer, `double * padfMaxBound` is actually a pointer to an array
> of 4 elements:

Great sleuthing! :)

This thread is a good example of "C's Biggest Mistake":


http://www.drdobbs.com/architecture-and-design/cs-biggest-mistake/228701625

Ali

On Saturday, 7 December 2013 at 23:11:39 UTC, Rémy Mouëza wrote:
> My strategy here would be to:
> A.  run the program in a debugger, say GDB, to get a exhaustive stacktrace for hints about where to look at.
> B. have a quick look at the library directly (the "Use the Source Luke" strategy).
>
> Since I was curious about your problem (you had everything correct - this should not fail), and have no access to your code, I checked out the shapelib code from its cvs public repository and found out that the last pointer, `double * padfMaxBound` is actually a pointer to an array of 4 elements: in shapelib/shpopen.c: SHPGetInfo: starting line 823:
>     for( i = 0; i < 4; i++ )
>     {
>         if( padfMinBound != NULL )
>             padfMinBound[i] = psSHP->adBoundsMin[i];
>         if( padfMaxBound != NULL )
>             padfMaxBound[i] = psSHP->adBoundsMax[i];
>     }
>
> This also explains why it does not segfault when you pass a null pointer: no array out of bounds happen then.
>
> padfMaxBound should point to a `double []` or `double [4]` instead of a mere `double`.
>
> I also ended checking the API documentation (http://shapelib.maptools.org/shp_api.html) and they do document that padfMinBound and padfMaxbound are pointers to 4 values X, Y, Z and M in a four entry array.
> You may want to check if there isn't any other of those "C tricks" hiding in your D bindings.

Good catch.  Now that you pointed it out I cannot for the life of
me figure out why I didn't think to check that!  To make matters
worse I didn't even have to go check the CVS, because I compiled
the library from source and had all the C code sitting just a
couple of 'cd's away.  For some reason it never occurred to me
that it might be expecting an array - even though one might
reasonably assume that the bounds on a multidimensional dataset
would have more than one dimension.

Thank you.


>
>
>
> On 12/07/2013 04:29 PM, Craig Dillabaugh wrote:
>> Hello,
>> I recently wrote bindings to the C-library Shapelib (it reads/writes a
>> common file format used in Geographic Information Systems).
>>
>> I've been trying to write a small test program to make sure my bindings
>> 'work' and I've come across a bizarre memory bug.  I THINK I've
>> identified the code that causes the problem, but I have no idea why.
>>
>> My test-suite includes this function:
>>
>> void shapeRead(string filename) {
>>   SHPHandle hShp = SHPOpen( std.string.toStringz( filename ), "rb" );
>>
>>   int n, shp_type;
>>   double pad_min_bound, pad_max_bound;
>>
>>   SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, &pad_max_bound);
>>
>>   SHPClose( hShp );
>> }
>>
>> If I comment out the SHPGetInfo call, then the segmentation fault
>> doesn't happen, but if its there then the program segfaults AFTER the
>> shapeRead function exits (eg. SHPClose runs fine) ).
>>
>> In fact if I call SHPGetInfo as follows, the crash doesn't occur:
>> SHPGetInfo( hShp, &n, &shp_type, &pad_min_bound, null); //NULL pointer
>> last arg.
>>
>> So in C the function SHPGetInfo is:
>>
>> void SHPAPI_CALL
>>       SHPGetInfo( SHPHandle hSHP, int * pnEntities, int * pnShapeType,
>>                   double * padfMinBound, double * padfMaxBound );
>>
>> While my D binding is (pretty much the same):
>>
>> extern( C ) void SHPGetInfo( SHPHandle hSHP, int* pnEntities,
>>       int* pnShapeType, double* padfMinBound, double* padfMaxBound );
>>
>>
>> I have no idea what is going on.  The sizes of ints and doubles are 4
>> and 8 bytes using gcc on my system (which is how I compiled my C
>> library) so those match the sizes of the corresponding D types [I
>> thought maybe there was a type-size mismatch and that was causing
>> something to be overwritten, but it doesn't appear that way].
>>
>> Any hints on where to look next would be appreciated.
>>
>> Craig

On Saturday, 7 December 2013 at 23:35:49 UTC, Ali Çehreli wrote:
> On 12/07/2013 03:11 PM, Rémy Mouëza wrote:
>
> > the last pointer, `double * padfMaxBound` is actually a
> pointer to an array
> > of 4 elements:
>
> Great sleuthing! :)
>
> This thread is a good example of "C's Biggest Mistake":
>
>
> http://www.drdobbs.com/architecture-and-design/cs-biggest-mistake/228701625
>
> Ali

If by "C's Biggest Mistake" you mean "Craig's Biggest Mistake"
you are incorrect. As hard as it may be to imagine I've done even
dumber things:o)

It is too bad that when I do do something dumb I tend to post about it on the internet!

Cheers

On 12/08/2013 12:16 AM, Craig Dillabaugh wrote:

> I cannot for the life of me figure out why I didn't think to check that!

D has already ruined your mind! :p

Ali