Google Summer of Code 2019, with Python Software Foundation
Python GSoC: Build a multi-user Blogging Platform with additional
features to manage GSoC@PSF
Sounak Pradhan
Matthew Lagoe, John ‘Warthog9’ Hawley, James Lopeman
Abstract
Every year more than 100 students apply for GSoC under the umbrella
organization PSF. Currently there is a multi-user blogging website
using WordPress CMS hosted for the students to publish their weekly
blogs and a static landing page for reaching out to people for more
information about this program. This project aims to build a
platform which allows smooth management of the GSoC program at PSF
every year and also ties everyone associated with it to PSF, so that
their work or they themselves can help out others in future.
Read Full Proposal
Important Links
Repositories:
python-gsoc/python-blogs,
python-gsoc/python-gsoc.github.io
Live Sites:
python-blogs,
python-blogs-static
GSoC Blog:
sounak98’s Blog
Summer Rewind
Let’s rewind to the beginning of this year. We had started working on
this application way before GSoC had even started. The goal was to
have a working application which PSF will be able to use for this
year’s GSoC for the management of their students. In this way, we will
be able to make sure that students actually use it and we get a clear
idea if the application is serving its purpose. I’m glad the plan
worked out, because tons of bugs were reported and we could fix them.
We also received valuable feedback from all users.
Schedulers and Builders
Allow me to introduce you to some of the most important modules of our
system. Without any doubt the first on the list is our
Scheduler
, which can perform particular tasks from
sending an email to archiving webpages. Well, the most powerful
feature of this module is that it can perform those tasks at any
particular date and time. Need to remind students that they have not
written a blog on time? Not a big deal, Scheduler
can do
that for you. Now think of this, many students who are not like me
publish their blogs on time, so we don’t really need to spam them with
emails. Thus we built the module Builder
which in turn
builds Scheduler
on different conditions.
Blogging Platform
We didn’t have to create a blogging platform as we integrated
aldryn-newsblog
but we had to tweak it a lot to fit into our system. Something that we
had to work on was setting up custom permissions for each user so that
they only have access to their blogs. We achieved this with the help
of django admin which allows us to set add, view, change permissions
based on querysets! Sanitizing the artilce contents was another
challenge that we faced because aldryn-newsblog uses an editor which
injects HTML
so that users can customize their blog
posts. Our system currently allows only particular tags like
<p>
, <h1>
, etc. Other tags are
sanitized conditionally, like for iframes we only render iframes for
YouTube videos so that users can add YouTube videos to their articles.
We also tweaked the article list templates to include our own
reddit-styled comment system which makes use of recursion on django
templates to display different threads. For more information on how to
achieve this, check out the article
Creating Reddit Styled Comment System with Django .
Tweaking Django
We used django-forms
wherever we could, but there were
cases where we needed to customize the forms to an extent which wasn’t
supported out of the box. We have a form which lets suborg admins and
admins add selected students to the system. Generally, this includes
adding a lot of students (~50). Typing out the emails one by one is
still okay, but selecting the GSoC year or the Suborg one by one for
each student? Ask my mentor, and he will let you know what a pain it
is. So, we tweaked the django form to add buttons which would let the
admin select a particular Suborg, year for all the mentioned users.
We have RSS feeds for each blog separately (all the articles published
by a student) and also for all the articles published on this
platform. Django has
Syndication Feed Framework
which allows customizable RSS feeds, but the all articles feed was too
long and took seconds to load. We needed to paginate the feed, and we
were out of luck as django didn’t support this out of the box. This
was a challenge, as we had to take in the request object and parse the
url to get the page number and render the blogs accordingly. We also
added the year argument which takes in the year and displays the blogs
of that particular GSoC. The current feed url looks something like
this
https://blogs.python-gsoc.org/en/feed/?y=2019&p=2.
For most of the other admin features, we heavily relied upon django
admin. The admin portal lets admins
- Add new
Scheduler
s and Builder
s
-
Check blog post histories of different articles to track changes
-
Add the current GSoC timeline which gets pushed to the schedule page
in the github site repository
- Send custom emails to users as admins
- Review comments and delete them if necessary
- Disable a user profile if the student fails mid-way
Integration with Github
We annotated some of the manual work that an admin has to put in to
maintain the static site on Github. Our system creates pull requests
adding new Suborgs in the Ideas page whenever a new Suborg Application
is added, it also archives current pages when the GSoC program ends.
These pulls can then be reviewed and merged to master by the admin.
Fixing Bugs
This was really a major part of the whole summer and it went hand in
hand with the whole developmene process. There were bugs that were
found by the users, and the others figured out by the mentors and me.
There are a ton of “Bugfix” PRs which were basically bug fixes.
There was a time when we pushed some changes and it made the system
send emails to all the users regardless of whether they have blogged
or not. Yes, basically we spammed a lot of users unintentionally. This
was another challenge that we had faced and overcame eventually by
making a flag which would disable all notifications to any user. We
also followed a strict push cycle to avoid any disturbance to students
blogging at the end of the week.
Wrapping Up
We ran accessibility tests on our websites and fixed issues which
decrease the accessibility of the website, like fixing contrast ratios
of texts and background, adding alts to images, etc. We also worked on
boosting the loading speed of out website on mobiles. In built tools
provided by Chrome and Firefox gave us a list of issues after
analyzing the website which we could work on.
We also ended up using a cache server to cache the data to fasten up
the whole loading process. We also needed to manually override caching
in some pages like the comments page, which would not show the new
comment as the old one was cached. This is the
issue
which describes more about this bug and how we solved it.
Future Plans
Currently, the platform provides most of the functionality required
for a smooth GSoC run at PSF, but there are features that would make
it even smoother for the admins and make their lives a bit less
painful. One of them is adding the mentors to the
GSoC site
automatically from the system’s database. This can be another nasty
manual work (typing in the emails, names, etc. one by one) and needs
to be done automatically. For more details, check this
issue
out.
We also need to write unit and integration tests for features that are
not provided by django or any third-party packages.
I would love to work on these in future even when this GSoC ends, fix
more bugs as and when they come up and be a part of this great
community!
Credits
First of all, any of this would not have been possible without my
mentors and other members of the PSF community. So a huge shout-out to
them for helping me whenever I needed and for guiding me when I was
clueless on how to proceed. While I was busy coding, my mentor would
look for potential bugs in the system and point them out to me. This
really kept me busy throughout the summer as I always had bugs to fix,
and helped me make the system more stable.
Next, I would thank Google for organising such an amazing program for
students who are passionate about coding and giving them an
opportunity to gain some hands-on experience.
Last but not the least, I would thank my fellow applicants who also
worked on building the application with me to bring it to a stage
where it could be used in this year’s GSoC run.
Apologies
There are a lot of mistakes that I made and learnt from them. In the
beginning I was not testing stuff through before making a PR,
sometimes trying to do things faster and the other times just being
lazy. It only made me spend more time on a particular feature as there
were things that would not work.
Another thing that I should apologize for is being very irregular
about posting blogs. This shouldn’t have come from me, as I was the
student working on the very blogging platform itself.
List of merged PRs
Here is a list of all the PRs made by me which got merged during the
application process and the summer.
-
#343
Article feed
-
#342
change color of rss feed icon
-
#341
Article feed
-
#340
Add year in feed links
-
#339
Add RSS button on blogs list and article list pages
-
#338
Minor fix
-
#337
Paginate feed
-
#335
Cache
-
#334
Cahce feed contents
-
#333
Change protocol in feed and sitemap
-
#332
Upgrade to django cms 3.7.0rc1, YES finally!
-
#331
Minor update
-
#330
Add modal when archiving pages
-
#329
Bugfix
-
#326
Add allowed styles
-
#325
Update requirements
-
#324
Add sanitizer
-
#321
Add never_cache to new_comment view
-
#318
Enhance user invite feature
-
#317
Logging
-
#315
bugfix
-
#310
Handle exception on sitemap
-
#309
Sitemap fix
-
#305
Add titles and meta tags
-
#304
Reglink notif not sent when suborg accepted
-
#303
Autocreate dir
-
#302
Upload attachment/image ckeditor
-
#301
Minor fix
-
#300
Add activation_date to SendEmail
-
#299
Send Email
-
#297
Redirect /blogs to /
-
#296
Archive old gsoc pages
-
#295
Reminder subject according to category of blog post
-
#285
Bugfix
-
#282
Add accepted blog in suborg details admin page
-
#279
Fix typo
-
#278
Fix pep8 warnings
-
#273
Bugfix
-
#271
Send notification to admin when suborg application added/updated
-
#270
Fix pep8 warnings
-
#269
Revoking student blog permissions after gsoc ends
-
#259
Fix suborg form validation condition
-
#258
Set update_site_template as custom handler
-
#255
Auto generate static site pages
-
#253
Fix url to access feed
-
#251
Fix typo
-
#250
Change builder admin
-
#249
Fix pep8 warnings
-
#248
Bugfixes
-
#246
Bugfix
-
#242
Add Suborg Application Form
-
#241
Suborg form
-
#239
Bugfixes
-
#237
Update models
-
#234
Update scope
-
#233
Decrease blog count only when article is created not updated
-
#232
Update calendar integration
-
#231
Minor changes
-
#230
Add import
-
#228
Add googleapiauth command
-
#223
Auto creates google calendar on adding events
-
#222
Update send reminder post blog logic
-
#220
Replace > and < with >, <
-
#219
Strip meta charset from articles
-
#217
Replace <pre> tag with >code< in articles
-
#216
Fix typo
-
#215
Fix typo
-
#213
Fix latest issues
-
#207
Add publish/unpublish article buttons
-
#206
Add mark reviewed button in articlereview admin
-
#205
Make blog url a hyperlink
-
#203
Extend fields and list_view for ArticleReviewAdmin
-
#200
Disable delete article except superusers
-
#198
Override clean function for wizard blog form
-
#197
Change blog main content field to lead_in
-
#194
Add button for reviewing blogs
-
#193
Fix runcron message display when no schedulers of current activation
…
-
#192
Separate reminder mails for students and mentors
-
#189
Add timeline
-
#188
Add feature for reminding students for blog posts
-
#186
Change laparams to None
-
#185
Bugfix
-
#181
Fix number of blogs in row
-
#177
Add email field in list display of userprofile
-
#176
Add error message if link exists
-
#175
Add reminders for reglinks
-
#173
Add email field to RegLink admin; alphabetically sort blogs
-
#172
Add used/all stat to add users admin page
-
#169
Comment delete, File upload fix
-
#168
Fix debug warnings
-
#166
Bugfix
-
#165
Auto publish blogs
-
#164
Bugfix
-
#162
Change queryset for blog
-
#161
Bugfix
-
#160
Fix minor bug
-
#158
Minor change
-
#157
Add proposal confirmation
-
#150
Fix add user bug and create recaptcha site key context processor
-
#146
Add timeline and gcal integration [WIP]
-
#143
Add Comments to Articles
-
#139
Move js to separate files
-
#137
Fix prospector warnings and migrate db
-
#136
Add error message when there are no blogs present
-
#134
Add error handling if sending email fails
-
#132
Add goto blog button for students
-
#125
Add proposal link to blogs list page
-
#123
Update db file
-
#122
Add attachment plugin to ckeditor
-
#120
Add page specific notifications
-
#118
Beautify README
-
#117
Add hidden field to UserProfile model to hide users
-
#113
Add error message when logging in using wrong credentials
-
#104
All blogs
-
#84
Revamp UI
-
#82
Fix typo and minor bug
-
#79
Student blog setup
-
#72
Add admin page for Scheduler model
-
#71
Deactivation user
-
#57
Add role field to UserProfile
-
#45
Extend djangocms_text_ckeditor
-
#32
Add runcron admin management command
-
#28
Add submenu support in sidemenu
-
#20
Add util function for sending messages in IRC channel
-
#18
Add filter for migrations
-
#12
Add template styling
Contact me at
sounak.98@gmail.com
if you have any queries regarding GSoC, PSF or python-blogs!
Google Summer of Code 2019, with Python Software Foundation
Python GSoC: Build a multi-user Blogging Platform with additional features to manage GSoC@PSF
Sounak Pradhan
Matthew Lagoe, John ‘Warthog9’ Hawley, James Lopeman
Abstract
Read Full Proposal
Important Links
Repositories: python-gsoc/python-blogs, python-gsoc/python-gsoc.github.io
Live Sites: python-blogs, python-blogs-static
GSoC Blog: sounak98’s Blog
Summer Rewind
Let’s rewind to the beginning of this year. We had started working on this application way before GSoC had even started. The goal was to have a working application which PSF will be able to use for this year’s GSoC for the management of their students. In this way, we will be able to make sure that students actually use it and we get a clear idea if the application is serving its purpose. I’m glad the plan worked out, because tons of bugs were reported and we could fix them. We also received valuable feedback from all users.
Schedulers and Builders
Allow me to introduce you to some of the most important modules of our system. Without any doubt the first on the list is our
Scheduler
, which can perform particular tasks from sending an email to archiving webpages. Well, the most powerful feature of this module is that it can perform those tasks at any particular date and time. Need to remind students that they have not written a blog on time? Not a big deal,Scheduler
can do that for you. Now think of this, many students who are not like me publish their blogs on time, so we don’t really need to spam them with emails. Thus we built the moduleBuilder
which in turn buildsScheduler
on different conditions.Blogging Platform
We didn’t have to create a blogging platform as we integrated
aldryn-newsblog
but we had to tweak it a lot to fit into our system. Something that we had to work on was setting up custom permissions for each user so that they only have access to their blogs. We achieved this with the help of django admin which allows us to set add, view, change permissions based on querysets! Sanitizing the artilce contents was another challenge that we faced because aldryn-newsblog uses an editor which injectsHTML
so that users can customize their blog posts. Our system currently allows only particular tags like<p>
,<h1>
, etc. Other tags are sanitized conditionally, like for iframes we only render iframes for YouTube videos so that users can add YouTube videos to their articles.We also tweaked the article list templates to include our own reddit-styled comment system which makes use of recursion on django templates to display different threads. For more information on how to achieve this, check out the article Creating Reddit Styled Comment System with Django .
Tweaking Django
We used
django-forms
wherever we could, but there were cases where we needed to customize the forms to an extent which wasn’t supported out of the box. We have a form which lets suborg admins and admins add selected students to the system. Generally, this includes adding a lot of students (~50). Typing out the emails one by one is still okay, but selecting the GSoC year or the Suborg one by one for each student? Ask my mentor, and he will let you know what a pain it is. So, we tweaked the django form to add buttons which would let the admin select a particular Suborg, year for all the mentioned users.We have RSS feeds for each blog separately (all the articles published by a student) and also for all the articles published on this platform. Django has Syndication Feed Framework which allows customizable RSS feeds, but the all articles feed was too long and took seconds to load. We needed to paginate the feed, and we were out of luck as django didn’t support this out of the box. This was a challenge, as we had to take in the request object and parse the url to get the page number and render the blogs accordingly. We also added the year argument which takes in the year and displays the blogs of that particular GSoC. The current feed url looks something like this https://blogs.python-gsoc.org/en/feed/?y=2019&p=2.
For most of the other admin features, we heavily relied upon django admin. The admin portal lets admins
Scheduler
s andBuilder
sIntegration with Github
We annotated some of the manual work that an admin has to put in to maintain the static site on Github. Our system creates pull requests adding new Suborgs in the Ideas page whenever a new Suborg Application is added, it also archives current pages when the GSoC program ends. These pulls can then be reviewed and merged to master by the admin.
Fixing Bugs
This was really a major part of the whole summer and it went hand in hand with the whole developmene process. There were bugs that were found by the users, and the others figured out by the mentors and me. There are a ton of “Bugfix” PRs which were basically bug fixes.
There was a time when we pushed some changes and it made the system send emails to all the users regardless of whether they have blogged or not. Yes, basically we spammed a lot of users unintentionally. This was another challenge that we had faced and overcame eventually by making a flag which would disable all notifications to any user. We also followed a strict push cycle to avoid any disturbance to students blogging at the end of the week.
Wrapping Up
We ran accessibility tests on our websites and fixed issues which decrease the accessibility of the website, like fixing contrast ratios of texts and background, adding alts to images, etc. We also worked on boosting the loading speed of out website on mobiles. In built tools provided by Chrome and Firefox gave us a list of issues after analyzing the website which we could work on.
We also ended up using a cache server to cache the data to fasten up the whole loading process. We also needed to manually override caching in some pages like the comments page, which would not show the new comment as the old one was cached. This is the issue which describes more about this bug and how we solved it.
Future Plans
Currently, the platform provides most of the functionality required for a smooth GSoC run at PSF, but there are features that would make it even smoother for the admins and make their lives a bit less painful. One of them is adding the mentors to the GSoC site automatically from the system’s database. This can be another nasty manual work (typing in the emails, names, etc. one by one) and needs to be done automatically. For more details, check this issue out.
We also need to write unit and integration tests for features that are not provided by django or any third-party packages.
I would love to work on these in future even when this GSoC ends, fix more bugs as and when they come up and be a part of this great community!
Credits
First of all, any of this would not have been possible without my mentors and other members of the PSF community. So a huge shout-out to them for helping me whenever I needed and for guiding me when I was clueless on how to proceed. While I was busy coding, my mentor would look for potential bugs in the system and point them out to me. This really kept me busy throughout the summer as I always had bugs to fix, and helped me make the system more stable.
Next, I would thank Google for organising such an amazing program for students who are passionate about coding and giving them an opportunity to gain some hands-on experience.
Last but not the least, I would thank my fellow applicants who also worked on building the application with me to bring it to a stage where it could be used in this year’s GSoC run.
Apologies
There are a lot of mistakes that I made and learnt from them. In the beginning I was not testing stuff through before making a PR, sometimes trying to do things faster and the other times just being lazy. It only made me spend more time on a particular feature as there were things that would not work.
Another thing that I should apologize for is being very irregular about posting blogs. This shouldn’t have come from me, as I was the student working on the very blogging platform itself.
List of merged PRs
Here is a list of all the PRs made by me which got merged during the application process and the summer.
Contact me at sounak.98@gmail.com if you have any queries regarding GSoC, PSF or python-blogs!