Initial Testing Plan for Plone

To come up with an initial testing plan, different approaches have been explored.

The first idea is to try to make Plone choke on some weird input in the forms or even by configuring Plone wrong on purpose.

It is also good to try to break stuff by simply removing it altogether to see how Plone reacts. This actually reveals what it is that the removed component is important for, and guides the writing of functional tests for it. As a side effect, a lot of thinking is provoked about how things are currently working, possibly revealing annoying hard-coded behaviour and pointing into directions for making the application more reliable or at least fail gracefully.

Unit tests are not sufficient in some use cases, and it sometimes even happens that we have to rewrite some unit tests as functional tests in order to have better coverage in face of Plone site customisations. In the best case these changes could always be fed back to customisation policies, but we have to test for the worst case, where in-depth changes for example to the permissions are made, and we need to ensure that there are functional tests that make problems directly evident.

Basic Content types tests

• These tests should best be placed in ATContentTypes, but to get started we can make adding and editing them part of our functional test suite:
• Document
• Event
• File
• Folder
• Image
• Folder
• Large Plone Folder
• Link
• News Item
• Topic Use cases:
  • Member adds one of the content types Member adds Favorite
  There are a few parts in Plone that treat specific content types in a special way, and these places should become evident in the functional tests. For example Links and Images, additional information is shown in plone_templates/folder_listing.pt We can take the test a step further and study what happens when you remove a content type altogether.

Portal Catalog

• The portal catalog contains indexes and metadata which would merit some special attention. Questions to address: What happens when the catalog is empty? What happens when you delete content in the ZMI find it through the catalog?

MailHost

• The MailHost is not a standard tool in the Plone sense, so it behaves just a bit differently. In addition, the MailHost is also a place where we are relying on a external service, so we have to take some extra care. One use case would be that there is a temporary problem with sending mail,

  and the administrator has to have a way to switch off the MailHost. He could remove it altogether, and once the problems are resolved, the MailHost is added back in again. This opens a feature request allowing to being able to desactivate a MailHost instead of having to delete it. Some questions we should address here:

  - What happens when we remove the MailHost altogether?

  • - Should related buttons and interfaces still be active?
    • - Can users still access the interface for sending mail? - Is prefs_mailhost_form still showing the configuration?
    - Create functional tests showing what breaks in Plone after removing the

    • MailHost, and then put it back in to see that the tests pass

  - What happens when the Mailhost is not correctly configured?

  - What happens when the MailHost can not send Mail? - Is there something that could have broken while moving from MailHost

  • to SecureMailHost?

Permissions

• Showing what effect the permission settings have is a great use of acceptance tests. It is nowhere documented what permissions are actually doing, and by coding this "knowledge" into functional tests, the practical meaning of permissions becomes evident. We should start with the permissions proper to Plone, and work our way down to permissions in CMF and Zope that are still relevant for Plone. As an example, we take the "Manage portal" permission of Plone. The following are all uses of this permission in Plone under the name "Manage portal". It does not take into account uses of the permission under the name

  CMFCorePermissions.ManagePortal

  -> Allows to see members that are not "listed" in the

  • portal_membership.searchForMembers()

  -> Allows to "Remove This Reply" in plone_content/discussionitem_view.pt -> Allows to search "Role(s)" in member_search_form.pt -> Allows "Remove This Discussion" in plone_templates/viewThreadsAtBottom.pt # General Zope permissions View = Permissions.view

  -> Mostly important in the workflow. See there for further information. -> Plays a role in the apis and on templates.

  AccessContentsInformation = Permissions.access_contents_information -> Mostly important in the workflow. See there for further information.

  DeleteObjects = Permissions.delete_objects

  -> Make sure deleting a content is only possible when you have the permission to delete the content itself, but also the permission to delete in the folder in general.

  ViewManagementScreens = Permissions.view_management_screens ManageProperties = Permissions.manage_properties FTPAccess = Permissions.ftp_access # CMF Base Permissions

  ListFolderContents = 'List folder contents' setDefaultRoles( ListFolderContents, ( 'Manager', 'Owner' ) ) -> Mostly important in the workflow. See there for further information.

  ListUndoableChanges = 'List undoable changes' setDefaultRoles( ListUndoableChanges, ('Manager',) ) # + Member

  AccessInactivePortalContent = 'Access inactive portal content' setDefaultRoles(AccessInactivePortalContent, ('Manager',))

  ModifyCookieCrumblers = 'Modify Cookie Crumblers' setDefaultRoles(ModifyCookieCrumblers, ('Manager',))

  ReplyToItem = 'Reply to item' setDefaultRoles(ReplyToItem, ('Manager',)) # + Member

  ManagePortal = 'Manage portal' setDefaultRoles(ManagePortal, ('Manager',))

  ModifyPortalContent = 'Modify portal content' setDefaultRoles(ModifyPortalContent, ('Manager',))

  ManageProperties = 'Manage properties' setDefaultRoles(ManageProperties, ('Owner','Manager',))

  ListPortalMembers = 'List portal members' setDefaultRoles( ListPortalMembers, ('Manager',) ) # + Member

  AddPortalFolders = 'Add portal folders' setDefaultRoles(AddPortalFolders, ('Owner','Manager')) # + Member

  AddPortalContent = 'Add portal content' setDefaultRoles(AddPortalContent, ('Owner','Manager',)) # + Member

  AddPortalMember = 'Add portal member' setDefaultRoles(AddPortalMember, ('Anonymous', 'Manager',))

  SetOwnPassword = 'Set own password' setDefaultRoles(SetOwnPassword, ('Manager',)) # + Member

  SetOwnProperties = 'Set own properties' setDefaultRoles(SetOwnProperties, ('Manager',)) # + Member

  MailForgottenPassword = 'Mail forgotten password' setDefaultRoles(MailForgottenPassword, ('Anonymous', 'Manager',)) # Workflow Permissions

  RequestReview = 'Request review' setDefaultRoles(RequestReview, ('Owner', 'Manager',))

  ReviewPortalContent = 'Review portal content' setDefaultRoles(ReviewPortalContent, ('Manager',)) # + Reviewer

  AccessFuturePortalContent = 'Access future portal content' setDefaultRoles(AccessFuturePortalContent, ('Manager',)) # + Reviewer

Plone Site Permission encapsulation

• There should be a way to encapsulate a Plone site so that it does not