Your SlideShare is downloading. ×

typemap in Perl/XS

649

Published on

for OSDC.TW 2013

for OSDC.TW 2013

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
649
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • # and of course, it usually doesn't.
  • # As you see, it's almost the same as declared in the .h files.
  • # so that a C compiler can compile.
  • # and should be.
  • # information on defines, includes, functions, typedefs, structs etc is stored in $converter's stash.
  • Transcript

    • 1. Wine label from Perl(located in Germany)Thanks , Wendy and Liz!
    • 2. ... and MST
    • 3. typemap in Perl/XSKenichi Ishigaki(charsbar)@OSDC.TW 2013April 19, 2013
    • 4. Let me ask youfirst.
    • 5. Have you ever writtenan extension thatbridges C and Perl?
    • 6. Part I: XS is ...?
    • 7. Perl has a feature to loadan external C library.use DynaLoader;my @paths = dl_findfile(...);my $lib = dl_load_file($path);
    • 8. This wont work correctlyunless the library conformswith Perls convention.use DynaLoader;my @paths = dl_findfile(...);my $lib = dl_load_file($path);
    • 9. So we need somethingin-between.Perl XS C library
    • 10. Writing the one-in-between is fairlyeasy.
    • 11. (as long as you dontadd anything extra)
    • 12. The simplest form of the one-in-between starts like this:#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
    • 13. These three lines are to usePerl API.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
    • 14. This is for portabilitybetween perls.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
    • 15. This is to import C functionsfrom the library.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
    • 16. Module and packagedeclarations.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
    • 17. The function declarations toexport (XSUBs) follow.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
    • 18. As with .h files, there shouldbe only declarations.#include "EXTERN.h"#include "perl.h"#include "XSUB.h"#include "ppport.h"#include <foo.h>MODULE = Foo PACKAGE = Foointfunc(const char* str)
    • 19. Nothing difficult(in principle).
    • 20. The library you usemay not always bethat simple.
    • 21. People may addsomething written in Cwith a variety of PerlAPI.
    • 22. If someones XScode scares you,please remember.
    • 23. Whats difficult isnot the XSinterface itself.
    • 24. Its something elsethat makes themattercomplicated.
    • 25. Dont hesitate writingXS interface if youfind a useful Clibrary.
    • 26. Part II: How .xs file is compiled
    • 27. To build an interfacelibrary, we need toturn its XS declarationinto pure C code.
    • 28. If you already haveMakefile.PL, run it, andthen, make.$ perl Makefile.PL && make
    • 29. If you have Build.PL, dosomething like this (ormaybe ./Build build).$ perl Build.PL && ./Build
    • 30. If you dont have either,Milla or Minilla will helpyou (ask miyagawa-san).
    • 31. The most important XSUB partwill be translated like this:XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
    • 32. A function exposed to the Perl worldtakes a Perl variable as its argument.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
    • 33. However, a Perl variable isactually a structure in C.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
    • 34. You cant pass it directly to a Cfunction, and vice versa.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
    • 35. The Perl variable is convertedand cast into a C value here.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
    • 36. Then, the function importedfrom the library is called.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
    • 37. And the return value is convertedinto a Perl value again.XS_EUPXS(XS_Foo_func){dVAR; dXSARGS;if (items != 1)croak_xs_usage(cv, "str");{int RETVAL;dXSTARG;const char* str = (const char *)SvPV_nolen(ST(0));RETVAL = func(str);XSprePUSH; PUSHi((IV)RETVAL);}XSRETURN(1);}
    • 38. typemap is thekey to thisconversion.
    • 39. Basic type mappingsare defined inExtUtils::typemap.$ perldoc -m ExtUtils::typemap
    • 40. # basic C typesint T_IVunsigned T_UVunsigned int T_UVlong T_IVunsigned long T_UVshort T_IVunsigned short T_UVchar T_CHARunsigned char T_U_CHARchar * T_PVunsigned char * T_PVconst char * T_PVcaddr_t T_PVwchar_t * T_PVwchar_t T_IV
    • 41. # bool_t is defined in <rpc/rpc.h>bool_t T_IVsize_t T_UVssize_t T_IVtime_t T_NVunsigned long * T_OPAQUEPTRchar ** T_PACKEDARRAYvoid * T_PTRTime_t * T_PVSV * T_SV
    • 42. Any type not listed thereshould be added locally,in a file named"typemap".
    • 43. Part III: On type mapping
    • 44. A typical"typemap" file hasthree sections.
    • 45. "TYPEMAP" section containsa pair of C type and XStype.TYPEMAPchar * T_PV
    • 46. How to map should be describedin the "INPUT" and "OUTPUT"sections, if necessary.INPUTT_PV$var = ($type)SvPV_nolen($arg)OUTPUTT_PVsv_setpv((SV*)$arg, $var);
    • 47. These code fragments will beinterpolated while processingan XS file.INPUTT_PV$var = ($type)SvPV_nolen($arg)OUTPUTT_PVsv_setpv((SV*)$arg, $var);
    • 48. Adding orphan C typesis easy.TYPEMAPmy_id T_IVmy_str T_PV
    • 49. Modifying thebehavior of anexisting type needssome consideration.
    • 50. If you are confident, justadd a new code fragment inyour local typemap.INPUTT_PVif (!SvOK($arg)) {$var = NULL;} else {$var = ($type)SvPV_nolen($arg);}
    • 51. Or, you might want to add anothertypedef in the XS file, beforeMODULE and PACKAGE declarations.typedef char * my_nullable_str;MODULE Foo PACKAGE FOOintfunc(my_nullable_str str)
    • 52. And then, add a behavior of thisnew type in your local typemap.TYPEMAPmy_nullable_str T_PV_OR_NULLINPUTT_PV_OR_NULLif (!SvOK($arg)) {$var = NULL;} else {$var = ($type)SvPV_nolen($arg);}
    • 53. There is also a muchtrickier way to do it.MODULE Foo PACKAGE FOOintfunc(char * str_or_null)
    • 54. If you dont get why thisworks, see perlref.INPUTT_PV$var = ${$var =~ /_or_null$/? qq{NULL}: qq{($type)SvPV_nolen($arg)}}
    • 55. Remove a star and prepend"OUT" (or "IN_OUT") for anargument called by reference.MODULE Foo PACKAGE FOOintfunc(char * str, OUT int len)
    • 56. Further Reading• perlxs• perlxstut• perlxstypemap• perlapi• perlcall• ... and others XS code :p
    • 57. Part IV: From .h filesto XS files
    • 58. Weve written XSfiles by hand so far.
    • 59. Of course this canbe done with a tool.
    • 60. The best-knowntool is h2xs.
    • 61. Its mainly used toexpose constantsfrom a C library.
    • 62. With the help ofC::Scan, you can use itto expose C functions aswell.$ h2xs -Axan Foo /path/to/header.h
    • 63. Defaulting to backwards compatibility with perl 5.xx.xIf you intend this module to be compatible withearlier perl versions, please specify a minimum perlversion with the -b option.Writing Foo/ppport.hScanning typemaps...Scanning /home/xxxx/perl5/perlbrew/perls/perl-5.xx.x/lib/5.xx.x/ExtUtils/typemapScanning /path/to/header.h for functions...Scanning /path/to/header.h for typedefs...Writing Foo/lib/Foo.pmWriting Foo/Foo.xsWriting Foo/typemapWriting Foo/Makefile.PLWriting Foo/READMEWriting Foo/t/Foo.tWriting Foo/ChangesWriting Foo/MANIFEST
    • 64. If the library is stable, this maygive you a good starting point.
    • 65. However.
    • 66. The files h2xs generatesare rather old-fashioned.
    • 67. Its not good at updatingonly a part of adistribution, either.
    • 68. I started writingConvert::H::XS.https://github.com/charsbar/convert_h_xs
    • 69. Convert::H::XS takes a C header file,and looks for the minimuminformation to write XS components.use Convert::H::XS;my $converter = Convert::H::XS->new;$converter->process($h_file);
    • 70. Convert::H::XS also providesmethods to write XS componentsfor convenience.$converter->write_constants("xs/constants.inc",);
    • 71. You can tweak thosepieces of informationwith a callback.$converter->write_functions("xs/functions.inc",sub {my ($type, $name, $args) = @_;...return ($type, $name, $args);});
    • 72. Grateful if you helpme improve this atthe hackathon.
    • 73. Part V: CPANTSupdate
    • 74. I talked aboutCPANTS last year.
    • 75. CPANTS tests what moduleauthors often forget to testby themselves.
    • 76. Perl QA Hackathon2013 in Lancaster
    • 77. Several metrics aregoing to be removed.
    • 78. Several metrics aregoing to be added.
    • 79. To add new metrics,we need to findissues.
    • 80. Issues are often raised byother QA/toolchain people.• Do not ship modules with Module::Install 1.04http://weblog.bulknews.net/post/33907905561/do-not-ship-modules-with-module-install-1-04• stop shipping MYMETA to CPANhttp://weblog.bulknews.net/post/44251476706/stop-shipping-mymeta-to-cpan
    • 81. We need to confirmthey are measurable,and widespread.
    • 82. What can I use toconfirm?
    • 83. • CPANTS databases• metacpan• CPAN grep
    • 84. I just wantedsomething else.
    • 85. Groonga• a fulltext search engine• a column store• actively developed (monthly release)• client/server (http/gqpt)• C API• Web interface
    • 86. Groonga::APIhttps://github.com/charsbar/groonga-api
    • 87. Also grateful if youhelp me improve thisat the hackathon.
    • 88. Thank you!

    ×