Django in the Office: Get Your Admin for Nothing and Your SQL for Free

2,440 views
2,357 views

Published on

Presenter: Raman Prasad
Abstract: Creating custom data models (similar to "content types" in Drupal) can quickly become complicated. Data that looks like it will fit into 1 database table might actually need 4 tables--or 34. Django, a python based web framework, excels as a method of creating relational database tables, providing comprehensive administrative pages for users, and pulling the data out again in a variety of formats.

Django may be used for large systems, but it's also suitable for small projects. This presentation will cover very basic Django models in the context of moving office data from spreadsheets to online databases.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,440
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Django in the Office: Get Your Admin for Nothing and Your SQL for Free

  1. 1. Django in the Office:Get Your Admin for Nothing and Your SQL for Free (Department of Molecular & Cellular Biology) 11.8.2011
  2. 2. “I wish I had a database”source: Bureaucratics by Jan Banning
  3. 3. custom websites/databases pre-2004/05 2005+
  4. 4. Django: python-based web framework source: http://xkcd.com/353/
  5. 5. Python and Django: Under-Hyped, Widely Used
  6. 6. Django: Scalable (without magic and superclouds) Firefox‟s Add-Ons Site Commenting System 250k add-ons 17,000 requests/second 150 million views/month 40 million users/month 500+ million api hits/day 450,000 websites (rewritten from CakePHP) 15 million profiles 75 million comments (stats from June 2011) (stats from 10/2010)
  7. 7. can start small: instrument scheduler(stats from July 2011. initial launch in jan/feb 2009)
  8. 8. Django: Maintenance/SecurityDjango - 2011 – 2 updates - 2010 – 2 updates - 2009 – 2 updates- Only update is the core Django code (python files) - No changes to your code - No changes to the database - No changes to HTML templates - No changes to static or uploaded files
  9. 9. Python and Django: Online and Local Support
  10. 10. Python: “For Dummies” - list >>> l = [cat, fish, 7, eats’, 108] >>> l.sort() >>> for x in l: ... print x 7 108 cat eats fish >>> len(x) 5
  11. 11. Python: “For Dummies” - dict >>> d = {1:Bob, 2:Susan, ‘buckle’:’shoe} >>> d.keys() [1, 2, ‘buckle’] >>> for k,v in d.iteritems(): ... print k, v.capitalize() 1 Bob 2 Susan buckle Shoe >>> print d.get(4, ‘not found’) not found
  12. 12. what about in the office? story of a 1-day, limited-use project (“raw example/bad naming, etc”)
  13. 13. The Request: Wednesday
  14. 14. Wednesday: The Request• 200+ trainees• 40 different labs or offices• 8 training dates• 2 types of training• 5 trainee classifications
  15. 15. Wednesday: The ResponseLet‟s make a database!
  16. 16. Wednesday: The ResponseLet‟s make a database!You can manage it through a website admin!
  17. 17. Wednesday: The ResponseLet‟s make a database!You can manage it through a website admin!Sortable column headers!
  18. 18. Wednesday: The ResponseLet‟s make a database!You can manage it through a website admin!Sortable column headers!You can have searching and filters!
  19. 19. Wednesday: The ResponseLet‟s make a database!You can manage it through a website admin!Sortable column headers!You can have searching and filters!We can send personalized emails with RSVP links!!
  20. 20. Wednesday: The ResponseLet‟s make a database!You can manage it through a website admin!Sortable column headers!You can have searching and filters!We can send personalized emails with RSVP links!!Just in case, you can download it all back to Excel!
  21. 21. Wednesday: “The Vision”
  22. 22. Wednesday: “Over Promising”
  23. 23. Thursday: Resource Constraints1 FTE dependent on the 77 bus
  24. 24. Thursday: Response -> Task List (9:30am)1- Let‟s make a database!2 - You can manage it through a website admin!3 - Sortable column headers!4 - You can have searching and filters!5 - We can send personalized emails with RSVP links!!6 - Just in case, you can download it all back to Excel!
  25. 25. a bit of setup: choose database (9:15am)In the settings.py file:DATABASES = { default: { ENGINE: django.db.backends.sqlite3, # Add postgresql_psycopg2, postgresql, mysql, sqlite3 or oracle. NAME: /db/mcb_hcom.db3, USER: , PASSWORD: , HOST: , PORT: , }}
  26. 26. a bit of setup: start an “app” (9:20am)>python manage.py startapp hcom_training
  27. 27. Data models: models.py (still 9:20am)/hcom_training/ models.py -> SQL tables! admin.py
  28. 28. a bit of setup: choose database (9:15am)In the settings.py file:INSTALLED_APPS = ( #... hcom_training, #...)
  29. 29. data models: think (9:30 to 9:40)What would C.J. Date do?
  30. 30. data models: 1st think (9:30 to 9:45) TrainingSession iname Trainee training type first name training datetime last name room email (etc, etc) location lab_or_office special confirmed_training_date id_md5 LabOffice (etc., etc.) name TraineeMessage SpecialDesignation trainee name time_sent to_address msg
  31. 31. data modelsclass Trainee(models.Model): """ Trainee Information """ fname = models.CharField(„First Name‟, max_length=50) lname = models.CharField(„Last Name‟, max_length=50) email= models.EmailField(blank=True) confirmed_training_date = models.BooleanField(default=False) location = models.ForeignKey(LocationInfo) # training session(+ 13 more fields)
  32. 32. data modelsclass LocationInfo(models.Model): """ Training Session Information """ name = models.CharField(max_length=255) training_datetime = models.DateTimeField() room = models.CharField(max_length=100)(+2 more fields)
  33. 33. models.py: create your tables (10:15am)> python manage.py validate0 errors found> python manage.py syncdbCreating tables ...Creating table hcom_training_specialCreating table hcom_training_labofficeCreating table hcom_training_locationinfoCreating table hcom_training_traineeCreating table hcom_training_traineemessageInstalling custom SQL ...Installing indexes ...No fixtures found
  34. 34. Thursday: Response -> Task List (10:02am)1- Let‟s make a database!2 - You can manage it through a website admin!3 - Sortable column headers!4 - You can have searching and filters!5 - We can send personalized emails with RSVP links!!6 - Just in case, you can download it all back to Excel!
  35. 35. models.py>python manage.py sqlall hcom_trainingCREATE TABLE `hcom_training_locationinfo` ( `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `name` varchar(255) NOT NULL, `training_type` varchar(100) NOT NULL, `training_datetime` datetime NOT NULL, ...CREATE TABLE `hcom_training_trainee` ( `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY, `fname` varchar(50) NOT NULL, `lname` varchar(50) NOT NULL, `email` varchar(75) NOT NULL, `confirmed_training_date` bool NOT NULL, `location_id` integer, `lab_or_office_id` integer NOT NULL, ...
  36. 36. 2 files: data models + admin/hcom_training/ models.py -> SQL tables! admin.py -> web-based admin, sortable columns, searching and filters
  37. 37. admin.py: 1 line of code (“You can manage it through a website admin!”)admin.site.register(Trainee)
  38. 38. admin.py: 1 line of code (“You can manage it through a website admin!”)admin.site.register(Trainee)
  39. 39. admin.py: 1 line of code (“You can manage it through a website admin!”)admin.site.register(Trainee)
  40. 40. Thursday: Response -> Task List (10:10am)1- Let‟s make a database!2 - You can manage it through a website admin!3 - Sortable column headers!4 - You can have searching and filters!5 - We can send personalized emails with RSVP links!!6 - Just in case, you can download it all back to Excel!
  41. 41. admin.py: 3 lines of code (“Sortable column headers!”)class TraineeAdmin(admin.ModelAdmin): list_display = (lname, ’fname, email, confirmed_training_date’, completed_training’)admin.site.register(Trainee, TraineeAdmin)
  42. 42. Thursday: Response -> Task List (10:15am)1- Let‟s make a database!2 - You can manage it through a website admin!3 - Sortable column headers!4 - You can have searching and filters!5 - We can send personalized emails with RSVP links!!6 - Just in case, you can download it all back to Excel!
  43. 43. admin.py: 4 lines of code (“You can have searching and filters!) class TraineeAdmin(admin.ModelAdmin): list_display = (lname, ’fname, email, confirmed_training_date’, completed_training’) search_fields = (lname, fname, email, ‘location_room’)admin.site.register(Trainee, TraineeAdmin)
  44. 44. admin.py: 5 lines of code class TraineeAdmin(admin.ModelAdmin): list_display = (lname, ’fname, email, confirmed_training_date’, completed_training’) search_fields = (lname, fname, email, ‘location_room’) list_filter = (confirmed_training_date, completed_training, location)admin.site.register(Trainee, TraineeAdmin)
  45. 45. admin.py: 5 lines of codeclass TraineeAdmin(admin.ModelAdmin): list_display = (lname, fname, email,confirmed_training_date, completed_training) search_fields = (lname, fname, email,location__room) list_filter = (confirmed_training_date,completed_training, location)admin.site.register(Trainee, TraineeAdmin)
  46. 46. Thursday: Response -> Task List (10:30am)1- Let‟s make a database!2 - You can manage it through a website admin!3 - Sortable column headers!4 - You can have searching and filters!5 - We can send personalized emails with RSVP links!!6 - Just in case, you can download it all back to Excel!
  47. 47. 2 Files: Models & Admin../hcom_training/ models.py -> SQL tables! admin.py -> web-based admin, sortable columns, searching and filters
  48. 48. admin.py -> just scratching the surfaceclass TraineeAdmin(admin.ModelAdmin): list_display = (lname,fname, email, active, confirmed_training_date,completed_training, special,approver_or_shopper, lab_or_office, hands_on_training_date, demo_training_date,) list_editable = (completed_training, ) search_fields = (lname, fname, email,) list_filter = (active, confirmed_training_date, completed_training,has_hands_on_training,has_demo_training,special,approver_or_shopper, location,) readonly_fields = ( has_hands_on_training, has_demo_training, last_update, id_md5,num_emails_sent,confirmation_link ) inlines = [ TraineeMessageInline, ] fieldsets = [ (name / email, {fields: [( fname,lname,), active, email, special,lab_or_office,last_update, num_emails_sent,]}) ,(training, {fields: [confirmed_training_date, completed_training, training_order,approver_or_shopper, location, (has_hands_on_training, hands_on_training_date,) , (has_demo_training, demo_training_date), ]}) ,(Notes, {fields: [ notes, id_md5 ,confirmation_link ]}) ]admin.site.register(Trainee, TraineeAdmin)
  49. 49. models.py -> shell -> ADD>python manage.py shell>>> t = Trainee(fname=Raman, lname=Prasad , email=rprasad@fas.harvard.edu)>>> t.save()>>> print t.id, t.lname, t.fname, t.email1457 Prasad Raman rprasad@fas.harvard.edu>>> t.delete()
  50. 50. models.py -> shell -> QUERY>>> l = Trainee.objects.filter(completed_training=False, special__name=Faculty)>>> l.count()23>>> for t in l:... print t.fname, t.lname(publicly listing tardy faculty members = bad idea!)
  51. 51. models.py -> shell -> EDIT>>> new_location = LocationInfo( name=special faculty training ,training_type=Hands-On Shopper , training_date=2011-11-15 , training_time=10:00 , room=NW 404)>>> new_location.save()>>> for t in l:... t.location = new_location... t.save()
  52. 52. models.py -> direct database access>python manage.py dbshellSQLite version 3.7.6Enter ".help" for instructionsEnter SQL statements terminated with a ";"sqlite>.tableshcom_training_labofficehcom_training_locationinfohcom_training_specialhcom_training_traineehcom_training_traineemessage
  53. 53. models.py -> dumpdata >python manage.py dumpdata hcom_training --indent=4{ "pk": 8, "model": "hcom_training.locationinfo", "fields": { "training_type": "Shopper Demo", "room": "Biolabs room 1080 (Lecture Hall)", "training_date": "2011-06-02", "training_time": "10:00:00", "week_num": "3", "name": "Shopper Demo: Thursday June 2nd 10am -12pm @ Biolabs room 1080 (Lecture Hall)" } }, { "pk": 1277, "model": "hcom_training.trainee", "fields": { "demo_training_date": "2011-05-20", "notes": "", "completed_training": true, "lab_or_office": 279, "confirmed_training_date": true, "has_demo_training": true, "last_update": "2011-05-23 10:39:14", "lname": "Akhmetova", "approver_or_shopper": "SHOPPER", "location": 3, "fname": "Laila", "active": true, "training_order": 14, "has_hands_on_training": false, "id_md5": "d759175de8ea5b1d9a2660e45554894f",
  54. 54. models.py -> but there‟s more! >python manage.py dumpdata hcom_training –-format=xml --indent=4{<object pk="9" model="hcom_training.locationinfo"> <field type="CharField" name="name">special faculty training</field> <field type="CharField" name="training_type">shopper</field> <field type="DateField" name="training_date">2011-11-15</field> <field type="TimeField" name="training_time">10:00:00</field> <field type="CharField" name="week_num"></field> <field type="CharField" name="room"></field> </object> <object pk="1277" model="hcom_training.trainee"> <field type="CharField" name="fname">Laila</field> <field type="CharField" name="lname">Akhmetova</field> <field type="BooleanField" name="active">True</field> <field to="hcom_training.special" name="special" rel="ManyToOneRel">356</field> <field type="CharField" name="email">lakhmet@fas.harvard.edu</field> <field type="DateField" name="hands_on_training_date"><None></None></field> <field type="BooleanField" name="has_hands_on_training">False</field> <field type="DateField" name="demo_training_date">2011-05-20</field> <field type="BooleanField" name="has_demo_training">True</field> <field type="BooleanField" name="confirmed_training_date">True</field> <field type="BooleanField" name="completed_training">True</field> <field to="hcom_training.locationinfo" name="location" rel="ManyToOneRel">3</field> <field type="CharField" name="approver_or_shopper">SHOPPER</field> <field to="hcom_training.laboffice" name="lab_or_office" rel="ManyToOneRel">279</field> <field type="IntegerField" name="training_order">14</field> <field type="TextField" name="notes"></field> <field type="DateTimeField" name="last_update">2011-05-23 10:39:14</field> <field type="CharField" name="id_md5">d759175de8ea5b1d9a2660e45554894f</field> </object>
  55. 55. models.py -> loaddata>python manage.py dumpdata hcom_training > hcom_2011_1106.json>python manage.py loaddata hcom_2011_1106.jsonInstalled 750 object(s) from 1 fixture(s)
  56. 56. Data In/Out- Web-based Admin-Python Shell or Python Scripts/Programs- dumpdata / loaddata (json, xml, yaml)- database shell / db specific commands (MySQL, sqlite, etc)
  57. 57. All for 2 files!!1- Let‟s make a database!2 - You can manage it through a website admin!3 - Sortable column headers!4 - You can have searching and filters!5 - We can send personalized emails with RSVP links!!6 - Just in case, you can download it all back to Excel!
  58. 58. personalized emails: messageDear Raman Prasad,You have been selected as an individual who will be placing orders via the Harvard Crimson OnlineMarketplace or "HCOM" procurement system on behalf of your lab or group.For REQUIRED Hands-on Shopper training, please save the SAVE THE DATE:Time: May 18th, 2011 at 10amLocation: Science Center, Rm. 226.The training will last 2 hours and lunch will be provided.To RSVP, please click on the following link: http://mcb.harvard.edu/hcom/confirm/2812e5cf6d8f21d69c91dddeefb792a7/If for any reason, you are not able to attend this training, please contact me immediately to makealternate arrangements.Thank you.
  59. 59. personalized emails: message as a templateDear {{ trainee.fname }} {{ trainee.lname }},You have been selected as an individual who will be placing orders via the Harvard Crimson OnlineMarketplace or "HCOM" procurement system on behalf of your lab or group.For REQUIRED Hands-on {{ trainee.training_type }} training, please save the SAVE THE DATE:Time: {{ trainee.location.training_date|date:"F jS, Y at P" }}Location: {{ trainee.location.room }}The training will last 2 hours and lunch will be provided.To RSVP, please click on the following link:{% url view_hcom_confirmation trainee.id_md5 %}If for any reason, you are not able to attend this training, please contact me immediately to makealternate arrangements.Thank you.
  60. 60. personalized emails: use the templatefor trainee in Trainee.objects.all(): msg = render_to_string(confirmation_email.txt’ , { trainee: trainee } )
  61. 61. personalized emailsfor trainee in Trainee.objects.all(): msg = render_to_string(confirmation_email.txt, { trainee: trainee } ) subject = Required HCOM Training at %s % trainee.location.name from_email =finance_office@mcb.harvard.edu‟ to_addresses = [ trainee.email ] bcc = [raman_prasad@harvard.edu] send_mail(subject, msg, from_email, to_addresses, bcc=bcc)Source: https://docs.djangoproject.com/en/dev/topics/email/
  62. 62. personalized emails: the linkhttp://mcb.harvard.edu/hcom/confirm/2812e5cf6d8f21d69c91dddeefb792a7/urlpatterns = patterns( hcom_training.views , url(r^hcom/confirm/(?P<trainee_id>(w){32})/$, view_hcom_confirmation , name=view_hcom_confirmation))
  63. 63. personalized emails: simplified viewdef view_hcom_confirmation(request, trainee_id): """Simple view showing trainee confirmation page""" trainee = Trainee.objects.get(id_md5=trainee_id) # get trainee trainee.confirmed_training_date = True # set confirmed to true trainee.save() # save the information # display web page return render_to_response(hcom_templates/hcom_confirm_page.html , { „trainee‟: trainee} , context_instance=RequestContext(request))
  64. 64. personalized emails: the viewdef view_hcom_confirmation(request, trainee_id): """Simple view showing trainee confirmation page""" lu = {} # Find the trainee; make sure he/she exists try: trainee = Trainee.objects.get(id_md5=trainee_id) lu.update({ trainee : trainee }) except Trainee.DoesNotExist: lu.update({ ERR_MSG : True, ERR_trainee_not_found : True }) returnrender_to_response(hcom_templates/hcom_confirm_page.html, lu, context_instance=RequestContext(request)) # Check if the person has already confirmed if trainee.confirmed_training_date: # Yes, err message lu.update({ ERR_MSG : True , ERR_already_confirmed : True }) else: # New confirmation, update database trainee.confirmed_training_date = True trainee.save() # show confirmation page to use returnrender_to_response(hcom_templates/hcom_confirm_page.html, lu, context_instance=RequestContext(request))
  65. 65. All for 2 files!!1- Let‟s make a database!2 - You can manage it through a website admin!3 - Sortable column headers!4 - You can have searching and filters!5 - We can send personalized emails with RSVP links!!6 - Just in case, you can download it all back to Excel!
  66. 66. export as Excelused python xlwt librarystyle_info_cell = easyxf(pattern: pattern solid, fore_colour white; align: wrap on;)book = xlwt.Workbook(encoding="utf-8") # make Excel workbooksheet1 = book.add_sheet(hcom training) # add a sheetrow_num = 2for trainee in Trainee.objects.all(): sheet1.write(row_num, 1, trainee.lname, style_info_cell) # lname sheet1.write(row_num, 2, trainee.fname, style_info_cell) # fname sheet1.write(row_num, 3, trainee.email, style_info_cell ) # email (…) row_num += 1Example: https://webapps.sciences.fas.harvard.edu/mcb/mcb-control-panel/hcom_training/
  67. 67. All for 2 files!!1- Let‟s make a database!2 - You can manage it through a website admin!3 - Sortable column headers!4 - You can have searching and filters!5 - We can send personalized emails with RSVP links!!6 - Just in case, you can download it all back to Excel!
  68. 68. Project Results- Emails sent out by 3:30/4pm after an hour of Q/ATime savings - RSVPs started coming. Database updated as email recipients clicked link - 35 people re-scheduled. No need to update shared spreadsheet. Staff used dropdown box in admin to change dates and add notes. - Able to send reminder emails before each training date
  69. 69. Project Results- Filters used to generate attendance spreadsheets- Tracked “completed training” via the admin- System used as new employees come on board- Home on time- BUT: Should have been a 1.5 to 2 day project
  70. 70. Spreadsheets -> Databases- Genome Modification Center - 46 tables - 2,389 objects for 500+ projects- Life Sciences Course Tracker - 37 Tables - 1,293 objects including 798 semesters of course data
  71. 71. Spreadsheets -> DatabasesConclusion: Django Models + Admin can take you far….
  72. 72. Spreadsheets -> Databases -> ModelsBut there‟s more!!>python manage.py inspectdb <your existing db>
  73. 73. Learning more .. .Django official site – Work through the Tutorial!!https://www.djangoproject.com/Pythonhttp://python.orgDive into Pythonhttp://www.diveintopython.net/toc/index.htmlContactraman_prasad@harvard.edu
  74. 74. thank you

×