Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Brown bsdmag july


Published on

Jim Brown's article on BSD certification for July issue of BSD Magazine.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Brown bsdmag july

  1. 1. LET’S TALKWhat It TakesStarting and Running an Open Source CertificationProgram, Part IILast month, in the first article in this series, we discussed thePeople aspect of running an Open Source certification programsuch as the BSD Certification Group (BSDCG). We discussed thetypes of people you’ll need in your program- SMEs, Writers,Translators, Technical Experts, Managers, the Advisory Group,and your Psychometrician.T his month we’ll look into the Processes that are discussing US organizations here. Other countries have involved in setting up a certification program. different types of legal business entities, and you’ll need A certification program should be set up as a specialized expertise in these not just an informal group of people. If you are From the beginning, the BSDCG really wanted to helpgoing to charge money for your exam, you will need some the BSD community. Our focus was not so much onsort of organization to take money in, pay money out, and making money as it was on getting BSD systems morepay your taxes- hopefully more of the former than the widely adopted. We decided to incorporate as a US non-latter two. profit organization. The actual mechanics of doing that were fairly straightforward.Setting Up the Business If this is the way your group wants to go, you will needThere is always a flurry of activity in getting an Open to do the following:Source project started, and a certification program is noexception. Almost immediately people will be asking how • Elect a group of people to act as officers (CEO,to pay for the certification. Unless you plan on stuffing all VP, Treasurer) for your certification organization.the money in your mattress, you’ll want to set up a bank Any business, even a non-profit corporation, needsaccount. But a bank is going to want to know if you are officers- a group of people authorized to speak and actsetting up a personal account or a business account. So for the organization. Note that the US Internal Revenuethe first thing you’ll want to decide is how to get set up as Service (IRS) requires that a majority of incorporatinga business. officers must be US citizens. In our case, we started There’s a stigma against business in the Open Source with just 3 individuals, and 2 of the 3 were US which is unfortunate because when it’s done well, a • Your group will have to draft Articles of Incorporationbusiness actually helps an Open Source project. A business that identify you as a US non-profit corporation in aprovides a focal point for legal, financial, and organizational US state. It’s handy, but not required, to live in theissues. A random group of people has a more difficult time state where you register. (Delaware is often chosenexplaining their organizational bona fides to government for its business friendly environment, even thoughagencies, financial institutions, and companies looking to the business is not physically located there. If youdo business with your certification program. do that, you’ll need to contract with an agent that will The technical details of setting up a business can forward all legal correspondence to your physicalbe challenging. There are several types of business location.) This is a formal step and should beentities such as a for-profit corporation, non-profit reviewed by a lawyer and an accountant if possible.corporation, sole proprietorship, S-corporation, Limited Depending on the state involved there may be anLiability Partnership (LLP) and more. And we’re only initial registration cost, and yearly renewals. 46 07/2011
  2. 2. What It Takes – Starting and Running an Open Source Certification Program, Part II• Your group will need to draft and approve Bylaws – convince them otherwise, they remained convinced published rules that determine how the company will that we were a business league. operate, how decisions are made, who is authorized • Once you have your EIN, set up a business bank to handle money, deal with legal issues, vote on account. You’re now officially open for business. choosing a Board of Directors, vote on organizational Banks will charge monthly services fees to keep your issues, keep rules of order for meetings, and other account. If you’re fortunate enough to keep a hefty mundane issues. There are many examples of average daily balance, they will generally waive those corporate Bylaws to pick and choose from, but take fees, but be alert- they will often try to sell you extra care- your business will be bound to follow these services which you probably don’t need. rules once they are adopted, so choose wisely. As with most corporate issues, it’s helpful to have a It’s a very good idea to engage someone with lawyer review the Bylaws. bookkeeping experience for non-profits to keep track of• Once you are a registered business, you’ll need to your finances. get an Employer Identification Number (EIN) from You can do it yourself, but the tax rules for non-profits the IRS. You need this even if you don’t have any differ from those of for-profit companies and it’s worth employees. It’s a number the IRS, banks, financial getting someone who knows what they are doing. There services companies, and other business entities use are also commercial software packages that handle to identify your business. At a minimum, you’ll need it non-profit organizations. These will need updating for filing your yearly tax return. periodically, as tax laws change frequently, even for non-• In the US, once you decide to incorporate as a non- profits. profit, you also have to decide what kind of non-profit you want to be. Because there are significant tax Setting Up the Certification advantages for donors, most non-profit organizations Getting the certification underway involves many separate, want to be recognized by the IRS as a 501(c)(3) interdependent tasks, a lot of which will depend on your charity organization- one that the IRS allows tax wider community. In our case, the BSD community has deductions for donors. This is another formal step been involved from the beginning in deciding what kind with a lengthy application process, and it will cost and number of certifications should be offered, what they money. Most of the effort is in documenting your should be named, what knowledge domains they should certification organization, its business plan, sources consider, and what kind of test delivery method should of funding, website, recent activities, public benefits, be available. In the beginning, the BSDCG set up mailing etc. If you are somehow fortunate to gain 501(c)(3) lists to give everyone a chance to express opinions status, you will have to eventually pass the Public (some more strongly than others) and exchange ideas. Support Test- at least 1/3 of your total support We also set up a live discussion channel, #bsdcert on (basically income and donations) normally comes from from the general public with corporate donors There were so many contrasting viewpoints and or sponsors subjected to certain limits although there differences of opinion on so many levels, it was very hard are some exceptions. to get rolling. Eventually, someone piped up and said that It turns out that the IRS uses a number of well we needed to do a Job Task Analysis (JTA) because this established precedents to evaluate new non- was needed to identify the specific skills a BSD system profit companies wishing to be a 501(c)(3) charity administrator needs on the job. It was a requirement for a organization. The general idea is that the organization psychometrically valid exam. This was the first time many must benefit the public as a whole, not just a subset of us had heard anything about psychometrics and was a of like-minded individuals. One precedent is for a key turning point. medical certification organization that looks very A JTA identifies all the important tasks that are similar to the BSDCG. The IRS determined that regularly performed by the position being certified (in our organization is really a 501(c)(6) organization, known case a BSD system administrator). It also identifies how as a Business League (described as an association often those tasks are performed. These tasks are then of persons having some common business interest), organized into domains of knowledge that are expected for which donations are not tax deductible. Since that to be known by candidates seeking certification. In our was an established precedent, they applied it to the case, someone stepped up and helped us put together BSDCG. While they listened politely while we tried to an online survey for the first Job Task Analysis 47
  3. 3. LET’S TALKBSD system administrators. We had several thousand are designed only to line the pockets of the certificationrespondents. company. Our psychometrician helped us refine the results into So before you set a price for your certification exam,our core knowledge domains which were published consider the following:as the official BSD Associate (BSDA) exam objectivesdocument ( • You’ll need to recover all your costs to at least breakpr_20051005_certreq_bsda_en_en.pdf). even. Factor in everything you have spent money A meeting of Subject Matter Experts (SMEs) was on so far. Include capital expenses (CapEx) likescheduled and a large number of questions (items) computers to host your item database, certificates,were developed and reviewed at the meeting. The printers for certificates, seals, etc. CapEx items arepsychometrician gave directions on how to score items tangible- things you can hold, view, or use. Operatingfor acceptability. The result was a large collection of pretty expenses (OpEx) are generally service relatedgood items for the BSDA exam. The next task was to expenses as discussed in the next which items from the pool were to be included on • Consider your ongoing OpEx items such as startupthe initial exam form (collection of items). costs including registration, legal, and accounting fees. We held these exam items in a standalone database Include costs for web hosting and maintenace fees.not accessible from the Internet. The reason was simple- Also include the delivery costs (see below), such asif that host was compromised, and the items published on exam printing and shipping. Shipping can be a largethe Net, all our work would have been in vain. expense over time if you are shipping exams packs, We were now ready to test the exam. and certificates overseas. Don’t forget advertising- you’ll want people to know about your exam. Advertising inTesting the Exam industry trade magazines is one effective way to do that,The next step was a Beta Test. We needed to know if the but there are other, lower cost methods such as userquestions were too easy, too hard, not appropriate, or groups, conference talks, and blogs.not understandable for non-native English speakers. We • Ongoing psychometric costs (OpEx). This can beselected 100 beta testers (the minimum recommended by substantial if you intend to maintain the certificationour psychometrician) and held a Beta exam. By offering a over time. Each certification program you runreduced price for the production (real) exam, we were able to should be annually reviewed to determine if anyget many Beta testers. This was a significant plus for us. items need to be added or subtracted. If (when) you We decided that the Beta exam would be distributed meet with your psychometrician in person, allow forby the paper-and-pencil exam method. The exam was reimbursement of travel costs. They are unlikely todelivered, and the results tabulated. During this time, travel on their own dime.our psychometrician pored over the results and gave us • If you raise money in some other way than examextensive assistance on selecting the correct items for sales, such as setting up a corporate sponsorshipthe official draft of the exam, as well as an analysis of the program, selling training materials, CDs, DVDs,computations for the cut score i.e. the minimum passing books, etc., include that money in your calculations.score. You’ll also want to look into your crystal ball to see howOther Considerations many candidates will take your exam. You’ll need toIf you’ve gotten this far, you are about to launch the official estimate wisely here, as overestimating exam incomeexam, but there are still several items you need to consider. can result in very strained financial situations.You still need to set a price for the exam, figure out how Finally, figure out your total income and expenses,to accept money via credit card, PayPal, checks, or even factor in a modest fudge factor, and divide by the numbercash; arrange one or more delivery methods, and figure of exam candidates you expect to definitely pay for andout some way to communicate to the candidate whether take the exam in one year. The result is your initial examthey passed or failed. We’ll discuss certificates and delivery price. If it seems too high (or too low) recheck your figuresmethods below. First, it’s time to consider the costs. and assumptions. Whatever it is, remember that this is Hopefully, you’ve already discussed how much to your initial price, and will likely change as your programcharge for the exam within your group. If so, you’ve heard matures.all the complaints about exorbitant fees for exams. It’s true The BSDCG went through the exact process above.that some certification exam fees are way out of line and Our goal was to produce an exam that is affordable 48 07/2011
  4. 4. by anyone regardless of where they are from. We metthat goal by setting our initial exam price at $75 USD in2008. It’s time to consider creating your certificates. Considerthe certificate carefully- a certificate will be visible to thesuccessful candidate (and everyone else) for severalyears. You want to project success and professionalism,so you should design an attractive looking certificate outof high quality materials. There are many examples tochoose from. We chose a simple, elegant design for theBSDA certificate. We also designed a 1-color embossedgold seal for the certificate that is applied individuallyto each certificate. And unlike many certificationorganizations, the signatures on a BSDA certificate arereal on every single one. Now for the delivery question- how will you actuallydeliver the exam? You’ve spent a long time developinga high-quality, psychometrically valid exam. If you justput it up on a web page with a Submit button, you willlose all of that advantage. Why? Because there is noway to positively identify the actual test taker with anonline exam over the Internet. In the certification world,just like the academic world, people will pay their friendsto take the exam for them if they think it will help thempass. The only reliable way to ensure that the examcandidate is who they say they are is with a proctor.An exam proctor checks each candidate’s identification,and personally administers the exam to ensure there isno cheating. Arranging to deliver proctored exams can be done inseveral ways:• Do it yourself at industry conferences, user groups, and corporate training events. The advantage is that you have total control over the testing environment. The disadvantage is that you can only reach a small number of people at a time, and travel costs can be a concern.• Arrange for your exam to be given by a trusted proctor at user groups, corporate training events, or industry conferences. This is a good method if you can find reliable, honest proctors. The BSDCG uses this method and chooses proctors from the computer industry that already have an established reputation. Advantages- good control; disadvantages- still only reaching a small number of candidates.• Engage a commercial test delivery company such as Vue or Prometric. Advantages- this can greatly expand your exam availability; disadvantages- can be very expensive (over $50,000 per year) with some
  5. 5. LET’S TALKIt’s worth looking at more than just the two big names. wwwversion2/outside/PERgeneral.asp?menuID=2)The BSDCG engaged with Schroeder Measurement international standard. The accreditation recognizesTechnologies (SMT for a certification organizations that maintain best practicesfraction of the above cost and we’ve now expanded our including the above practices and offerings to over 300 test centers in some 20 To achieve this accreditation, your certification programcountries. The price for the computer based exam had joins the ANSI program and starts the clock on theto change to accommodate this arrangement, but even at accreditation. You are assigned two ANSI auditors- one$150 USD, the exam is still affordable to almost everyone. who does the initial audit and one for the final audit.The paper-and-pencil exam is still just $75 USD. The initial audit is where you find out all your governance An ideal proctor is an individual with no knowledge issues – documentation, management reviews, procedures,of the exam subject material. This means they couldn’t etc. that need corrective action. Once you make progressanswer any questions on the exam even if they wanted you schedule the second audit where they take an evento. While it’s not always possible to find an ideal proctor, closer look to make sure your program is fully compliant withyou can ensure a positive exam experience by drafting ANSI/ISO/IEC 17024. At the end of eighteen months, youa proctor guidelines document that discusses what the will either have succeeded or been placed on remediationproctor should do in setting up, delivering, wrapping up track where you have even more work to do to improve youran exam session, and returning all the materials. Also, all program.proctors should sign a Non-Disclosure Agreement, and More information on ANSI accreditation and the ANSI/agree to keep the contents of the exam materials private ISO/IEC 17024 standard can be found on their website.and safe at all times. It is expensive, but the benefits are worth it – you get access to international and government markets and youSetting Up for the Final Exam Launch get recognized as a best in class certification program.Finally, you are ready to launch. Now is the time to Get that and you’ll really have something to celebrate.engage your media team to get the word out about yournew exam. Tune up that advertising you’ve already been *************************************************************doing to announce your new certification. Get the wordout with blogs, user groups, social media and any other In next month’s article, we’ll finish up by discussing theway you can. Technology aspects of a certification program- surveys, Once it’s offically released, you’ll need to score the collaboration, test construction, delivery, scoring andexam and communicate the results. We’ll discuss these reporting, and security. We’ll also look at key metricsitems in next month’s article on the Technology elements that can be useful in the ongoing governance andinvolved in your certification program. administration of the program.GovernanceBefore you sit back and celebrate, you still havesome work to do. For the certification program to besuccessful, you have to have regular Board of Directorsmeetings to discuss any issues. You need to periodicallyreview the exam results with the psychometrician. Youshould also regularly review the finances to see whetheryou are making or losing money. You also need toregularly review contracts with your psychometricianand test delivery companies. And don’t forget to updateyour website. JIM BROWN But governance of your program is more than just a list Jim Brown has worked in the computer industry with continuousof activities. Since you’ve put a lot into the program and Unix involvement in development or administration since theyou want it to be the best it can be, consider looking into early 1980s. His experience includes applications, systemsaccreditation of your certification program by the American and database programming, in a variety of languages. One ofNational Standards Institute (ANSI the founders of the BSD Certi�cation Group, he is helping to They’ve developed an accreditation program based develop the BSD Professional certi�cation. He currently lives inon the ANSI/ISO/IEC 17024 ( Northwest Arkansas, USA. 50 07/2011