Your SlideShare is downloading. ×
  • Like
typemap in Perl/XS
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

typemap in Perl/XS

  • 598 views
Published

for OSDC.TW 2013

for OSDC.TW 2013

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
598
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
9
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • # and of course, it usually doesn't.
  • # As you see, it's almost the same as declared in the .h files.
  • # so that a C compiler can compile.
  • # and should be.
  • # information on defines, includes, functions, typedefs, structs etc is stored in $converter's stash.

Transcript

  • 1. Wine label from Perl(located in Germany)Thanks , Wendy and Liz!
  • 2. ... and MST
  • 3. typemap in Perl/XSKenichi Ishigaki(charsbar)@OSDC.TW 2013April 19, 2013
  • 4. Let me ask youfirst.
  • 5. Have you ever writtenan extension thatbridges C and Perl?
  • 6. Part I: XS is ...?
  • 7. Perl has a feature to loadan external C library.use DynaLoader;my @paths = dl_findfile(...);my $lib = dl_load_file($path);
  • 8. This wont work correctlyunless the library conformswith Perls convention.use DynaLoader;my @paths = dl_findfile(...);my $lib = dl_load_file($path);
  • 9. So we need somethingin-between.Perl XS C library
  • 10. Writing the one-in-between is fairlyeasy.
  • 11. (as long as you dontadd anything extra)
  • 12. The simplest form of the one-in-between starts like this:#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
  • 13. These three lines are to usePerl API.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
  • 14. This is for portabilitybetween perls.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
  • 15. This is to import C functionsfrom the library.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
  • 16. Module and packagedeclarations.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
  • 17. The function declarations toexport (XSUBs) follow.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
  • 18. As with .h files, there shouldbe only declarations.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
  • 19. Nothing difficult(in principle).
  • 20. The library you usemay not always bethat simple.
  • 21. People may addsomething written in Cwith a variety of PerlAPI.
  • 22. If someones XScode scares you,please remember.
  • 23. Whats difficult isnot the XSinterface itself.
  • 24. Its something elsethat makes themattercomplicated.
  • 25. Dont hesitate writingXS interface if youfind a useful Clibrary.
  • 26. Part II: How .xs file is compiled
  • 27. To build an interfacelibrary, we need toturn its XS declarationinto pure C code.
  • 28. If you already haveMakefile.PL, run it, andthen, make.$ perl Makefile.PL && make
  • 29. If you have Build.PL, dosomething like this (ormaybe ./Build build).$ perl Build.PL && ./Build
  • 30. If you dont have either,Milla or Minilla will helpyou (ask miyagawa-san).
  • 31. The most important XSUB partwill be translated like this:XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
  • 32. A function exposed to the Perl worldtakes a Perl variable as its argument.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
  • 33. However, a Perl variable isactually a structure in C.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
  • 34. You cant pass it directly to a Cfunction, and vice versa.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
  • 35. The Perl variable is convertedand cast into a C value here.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
  • 36. Then, the function importedfrom the library is called.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
  • 37. And the return value is convertedinto a Perl value again.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
  • 38. typemap is thekey to thisconversion.
  • 39. Basic type mappingsare defined inExtUtils::typemap.$ perldoc -m ExtUtils::typemap
  • 40. # basic C typesint T_IVunsigned T_UVunsigned int T_UVlong T_IVunsigned long T_UVshort T_IVunsigned short T_UVchar T_CHARunsigned char T_U_CHARchar * T_PVunsigned char * T_PVconst char * T_PVcaddr_t T_PVwchar_t * T_PVwchar_t T_IV
  • 41. # bool_t is defined in <rpc/rpc.h>bool_t T_IVsize_t T_UVssize_t T_IVtime_t T_NVunsigned long * T_OPAQUEPTRchar ** T_PACKEDARRAYvoid * T_PTRTime_t * T_PVSV * T_SV
  • 42. Any type not listed thereshould be added locally,in a file named"typemap".
  • 43. Part III: On type mapping
  • 44. A typical"typemap" file hasthree sections.
  • 45. "TYPEMAP" section containsa pair of C type and XStype.TYPEMAPchar * T_PV
  • 46. How to map should be describedin the "INPUT" and "OUTPUT"sections, if necessary.INPUTT_PV$var = ($type)SvPV_nolen($arg)OUTPUTT_PVsv_setpv((SV*)$arg, $var);
  • 47. These code fragments will beinterpolated while processingan XS file.INPUTT_PV$var = ($type)SvPV_nolen($arg)OUTPUTT_PVsv_setpv((SV*)$arg, $var);
  • 48. Adding orphan C typesis easy.TYPEMAPmy_id T_IVmy_str T_PV
  • 49. Modifying thebehavior of anexisting type needssome consideration.
  • 50. If you are confident, justadd a new code fragment inyour local typemap.INPUTT_PVif (!SvOK($arg)) {$var = NULL;} else {$var = ($type)SvPV_nolen($arg);}
  • 51. Or, you might want to add anothertypedef in the XS file, beforeMODULE and PACKAGE declarations.typedef char * my_nullable_str;MODULE Foo PACKAGE FOOintfunc(my_nullable_str str)
  • 52. And then, add a behavior of thisnew type in your local typemap.TYPEMAPmy_nullable_str T_PV_OR_NULLINPUTT_PV_OR_NULLif (!SvOK($arg)) {$var = NULL;} else {$var = ($type)SvPV_nolen($arg);}
  • 53. There is also a muchtrickier way to do it.MODULE Foo PACKAGE FOOintfunc(char * str_or_null)
  • 54. If you dont get why thisworks, see perlref.INPUTT_PV$var = ${$var =~ /_or_null$/? qq{NULL}: qq{($type)SvPV_nolen($arg)}}
  • 55. Remove a star and prepend"OUT" (or "IN_OUT") for anargument called by reference.MODULE Foo PACKAGE FOOintfunc(char * str, OUT int len)
  • 56. Further Reading• perlxs• perlxstut• perlxstypemap• perlapi• perlcall• ... and others XS code :p
  • 57. Part IV: From .h filesto XS files
  • 58. Weve written XSfiles by hand so far.
  • 59. Of course this canbe done with a tool.
  • 60. The best-knowntool is h2xs.
  • 61. Its mainly used toexpose constantsfrom a C library.
  • 62. With the help ofC::Scan, you can use itto expose C functions aswell.$ h2xs -Axan Foo /path/to/header.h
  • 63. Defaulting to backwards compatibility with perl 5.xx.xIf you intend this module to be compatible withearlier perl versions, please specify a minimum perlversion with the -b option.Writing Foo/ppport.hScanning typemaps...Scanning /home/xxxx/perl5/perlbrew/perls/perl-5.xx.x/lib/5.xx.x/ExtUtils/typemapScanning /path/to/header.h for functions...Scanning /path/to/header.h for typedefs...Writing Foo/lib/Foo.pmWriting Foo/Foo.xsWriting Foo/typemapWriting Foo/Makefile.PLWriting Foo/READMEWriting Foo/t/Foo.tWriting Foo/ChangesWriting Foo/MANIFEST
  • 64. If the library is stable, this maygive you a good starting point.
  • 65. However.
  • 66. The files h2xs generatesare rather old-fashioned.
  • 67. Its not good at updatingonly a part of adistribution, either.
  • 68. I started writingConvert::H::XS.https://github.com/charsbar/convert_h_xs
  • 69. Convert::H::XS takes a C header file,and looks for the minimuminformation to write XS components.use Convert::H::XS;my $converter = Convert::H::XS->new;$converter->process($h_file);
  • 70. Convert::H::XS also providesmethods to write XS componentsfor convenience.$converter->write_constants("xs/constants.inc",);
  • 71. You can tweak thosepieces of informationwith a callback.$converter->write_functions("xs/functions.inc",sub {my ($type, $name, $args) = @_;...return ($type, $name, $args);});
  • 72. Grateful if you helpme improve this atthe hackathon.
  • 73. Part V: CPANTSupdate
  • 74. I talked aboutCPANTS last year.
  • 75. CPANTS tests what moduleauthors often forget to testby themselves.
  • 76. Perl QA Hackathon2013 in Lancaster
  • 77. Several metrics aregoing to be removed.
  • 78. Several metrics aregoing to be added.
  • 79. To add new metrics,we need to findissues.
  • 80. Issues are often raised byother QA/toolchain people.• Do not ship modules with Module::Install 1.04http://weblog.bulknews.net/post/33907905561/do-not-ship-modules-with-module-install-1-04• stop shipping MYMETA to CPANhttp://weblog.bulknews.net/post/44251476706/stop-shipping-mymeta-to-cpan
  • 81. We need to confirmthey are measurable,and widespread.
  • 82. What can I use toconfirm?
  • 83. • CPANTS databases• metacpan• CPAN grep
  • 84. I just wantedsomething else.
  • 85. Groonga• a fulltext search engine• a column store• actively developed (monthly release)• client/server (http/gqpt)• C API• Web interface
  • 86. Groonga::APIhttps://github.com/charsbar/groonga-api
  • 87. Also grateful if youhelp me improve thisat the hackathon.
  • 88. Thank you!