Home › Groups › NABUUR.com Website maintenance and development team › Project room

[Tech] making modules decision

Status: In progress
Step: 
3

Technical discussion about modules in use for the Drupal 6 site. Situation as it stands can be found on http://theme.nabuurtest.com/ and http://svn.nabuurtest.com/trac/

Task closes on final decision - live(!), if a new project is set up (might make sense) or if we move it to somewhere geeky like the tracker.

Project: Join the NABUUR Team
701 reads
stdClass Object ( [nid] => 256807 [vid] => 259207 [type] => task_discussion [status] => 1 [created] => 1261138864 [changed] => 1261138864 [comment] => 0 [promote] => 1 [sticky] => 0 [revision_timestamp] => 1261138864 [title] => Re: [Tech] making modules decision [body] => [c&p from http://www.nabuur.com/en/group/nabuurcom-website-development/project/task/join-nabuur-developers-team by frans] Hi Raul, Thanks for taking he time to have a look at the new website and the comments on it here. Let me give you some answers. Quote: Thanks for giving me access to the back end. Now I can give you my comments. - So far it looks pretty much like it is being built almost in the same way as the current site I am not sure what you mean here. Please don't forget we are dealing with an existing application. Nabuur.com has a long history in which it have developed a certain procedure of handling villages, projects and tasks. I don't have the feeling the nabuur community would like to fall back to just being a forum platform. On the other hand we have changed a lot in the new site: * New villages don need to have a project, just a window to the world and communicating is alright now. They have some time to orientate themselves, collect a community around them before starting a project. * On the village frontpage we have a "hang out" place where neighbours can communicate with small messages. We hope to have less "off topic" messages in the projects. * The process of starting a village on the current site is very cumbersome. We spend a lot of time to make this very easy in the new site. The theming of the villages pages is not ready yet. It will be finished before the NabuurCamp. A number of blocks are still to be add. Ideas to make it better are welcome. Quote: - There are way too many modules and a lot of them which I don't see a purpose to put. I do not add a module unless it is completely necessary and the value is adding is way higher than the speed is taking away from the site and specially in this type of website that is going to be used by people in countries with low speed. Just to give you an idea, it took about almost 30 seconds to load the modules page and that is with a speed of 3Mbb/s. You are right, don't use modules, unless you really need them. I would like to minimize this too. Some modules are only for developement, others are leftovers from the Drupal 5 site who can be deleted later. I am afraid that most modules are really needed if you want all functionality asked for in Nabuur.com. PS: Using the modules page for a speed test is an unlucky choice. Loading this page makes drupal to clear all database data about modules and menus, then it is looping through all modules to build up the new tables. Most of the 30 seconds was spend on the server, not on sending data over your high speed connection. Users will never have to load this page. Quote: - Just to give you some examples of modules that are taking too much speed and not adding a huge improvement * Apache solr - Is there really a need where you can set up a lot of specific searches using views? * Calais: Are we really into a stage to start using Calais given the level of development of people. I really don't see a purpose * Apache Solr is a search feature, wich is offloading database processing from the drupal database. While drupal faceted search, which we use in the current site is using the database quite heavy. Apache Solr is in the development environment as a test, to investigate the possibilities. * Calais: "given the development of people" we see that free taging on the current site is not workig at all (example. The Calais web service automatically attaches rich semantic metadata to the content you submit. Using natural language processing, machine learning and other methods, Calais categorizes and links your document with entities (people, places, organizations, etc.), facts (person “x” works for company “y”), and events (person “z” was appointed chairman of company “y” on date “x”). We hope the OpenCalais module can classify the stories on the site automaticaly and better. Users would not use OpenCalais at all, it will make live of the neighbours easier. This could be used in views with exposed filters. Quote: Theme The genesis theme you are using is not really a theme that is that easy to theme as compared with other more popular themes. At the same time, given the amount of resources invested in this site, why not to buy a theme for about 300Eur that could make it a lot easier to theme on a framework that is very developed. It is obvious right now that the theme is not looking very well and I also found some issues with code just on a very superficial look. I am not sure I agree with you here. Of course, when you start using a new base theme, there is a learning curve. We choose Genisis because it is flexible to configure custom regions, and it has its own gpanels technology. The idea was not to use the panels module (more on this later). About buying a drupal theme: you should first read this and another article. The theme is not looking great at the moment: because theming is not finished yet. Quote: Using the power of popular modules it would be great to give more power to modules like views that make it a lot easier for people to search content which is for example one of the things that are most important for people. Why not use more exposed filters to search content No question about this. Use views, deploy views where-ever you can. Quote: Why is panels not being used. One of the most powerful modules at the moment. We use panels in the current site, and it is a great module. There where a few reasons why we didn't want to use Panels in the new site: * Panels was not available for Drupal 6 when we started developing the new site. * Panels carries a performance hit because the layout variables live in the database and a lot of code is executed to render the pages. * Caching of panels and panes works fine on a static site, but is much more difficult in a community site where content is added every minute. * Genesis whith gpanels can do the trick. On the other hand, because of the new situation, the finishing of the new site will be done by volunteer developers. That could be a reason to use panels again. Some last remarks: * There is a lot of exsisting data. About 180.000 nodes, 17.500 files, 20.000 embedded videos, 50.000 external links and much more. People have put a lot of effort to write and upload this content, don't let it get lost. * There are a lot of long time functionalities and procedures like villages, projects, tasks, discussions, wiki's. The community has to decide it they want to loose any of these. * Most of the groups are just used as forums. Investigate if converting the groups to forums would be feasable and make the site easier. Migrating the current site to an easier to use new one is a big challenge. Hope this clears up the questions to have about the new website for Nabuur.com. [teaser] => [c&p from http://www.nabuur.com/en/group/nabuurcom-website-development/project/task/join-nabuur-developers-team by frans] Hi Raul, Thanks for taking he time to have a look at the new website and the comments on it here. Let me give you some answers. Quote: Thanks for giving me access to the back end. Now I can give you my comments. - So far it looks pretty much like it is being built almost in the same way as the current site [log] => [format] => 1 [uid] => 17035 [name] => kester [picture] => [data] => a:9:{s:8:"og_email";s:1:"2";s:16:"privatemsg_allow";i:1;s:28:"privatemsg_setmessage_notify";i:1;s:17:"messaging_default";s:4:"mail";s:7:"contact";i:0;s:29:"taxonomy_image_disable_images";i:0;s:16:"nodecomment_sort";s:1:"1";s:8:"sms_user";a:1:{i:0;N;}s:14:"tinymce_status";s:4:"true";} [path] => group/nabuurcom-website-development/project/task/discussion/re-tech-making-modules-decision [field_embedded_video] => Array ( [0] => Array ( [embed] => [value] => [provider] => [data] => Array ( ) ) ) [field_external_link] => Array ( [0] => Array ( [url] => [title] => [attributes] => N; ) [1] => Array ( [url] => [title] => [attributes] => N; ) ) [comment_type] => task_discussion [og_groups] => Array ( [0] => 14315 ) [og_groups_both] => Array ( [14315] => NABUUR.com Website maintenance and development team ) [og_public] => 1 [tags] => [nodewords] => Array ( ) [trid] => 0 [language] => en [i18n_status] => 0 [taxonomy] => Array ( ) [translation] => Array ( ) [cid] => 256807 [pid] => 0 [hostname] => 83.163.41.240 [thread] => 01/ [mail] => [homepage] => [comment_target_nid] => 256806 [depth] => 0 )
kester
Re: [Tech] making modules decision
Fri, 2009/12/18 - 14:21

[c&p from http://www.nabuur.com/en/group/nabuurcom-website-development/project/tas... by frans]

Hi Raul,

Thanks for taking he time to have a look at the new website and the comments on it here. Let me give you some answers.

Quote:

Thanks for giving me access to the back end. Now I can give you my comments.

- So far it looks pretty much like it is being built almost in the same way as the current site

I am not sure what you mean here. Please don't forget we are dealing with an existing application. Nabuur.com has a long history in which it have developed a certain procedure of handling villages, projects and tasks. I don't have the feeling the nabuur community would like to fall back to just being a forum platform.

On the other hand we have changed a lot in the new site:

* New villages don need to have a project, just a window to the world and communicating is alright now. They have some time to orientate themselves, collect a community around them before starting a project.
* On the village frontpage we have a "hang out" place where neighbours can communicate with small messages. We hope to have less "off topic" messages in the projects.
* The process of starting a village on the current site is very cumbersome. We spend a lot of time to make this very easy in the new site.

The theming of the villages pages is not ready yet. It will be finished before the NabuurCamp. A number of blocks are still to be add.

Ideas to make it better are welcome.

Quote:

- There are way too many modules and a lot of them which I don't see a purpose to put. I do not add a module unless it is completely necessary and the value is adding is way higher than the speed is taking away from the site and specially in this type of website that is going to be used by people in countries with low speed. Just to give you an idea, it took about almost 30 seconds to load the modules page and that is with a speed of 3Mbb/s.

You are right, don't use modules, unless you really need them. I would like to minimize this too. Some modules are only for developement, others are leftovers from the Drupal 5 site who can be deleted later. I am afraid that most modules are really needed if you want all functionality asked for in Nabuur.com.
PS: Using the modules page for a speed test is an unlucky choice. Loading this page makes drupal to clear all database data about modules and menus, then it is looping through all modules to build up the new tables. Most of the 30 seconds was spend on the server, not on sending data over your high speed connection. Users will never have to load this page.

Quote:

- Just to give you some examples of modules that are taking too much speed and not adding a huge improvement
* Apache solr - Is there really a need where you can set up a lot of specific searches using views?
* Calais: Are we really into a stage to start using Calais given the level of development of people. I really don't see a purpose

* Apache Solr is a search feature, wich is offloading database processing from the drupal database. While drupal faceted search, which we use in the current site is using the database quite heavy. Apache Solr is in the development environment as a test, to investigate the possibilities.
* Calais: "given the development of people" we see that free taging on the current site is not workig at all (example.
The Calais web service automatically attaches rich semantic metadata to the content you submit. Using natural language processing, machine learning and other methods, Calais categorizes and links your document with entities (people, places, organizations, etc.), facts (person “x” works for company “y”), and events (person “z” was appointed chairman of company “y” on date “x”).
We hope the OpenCalais module can classify the stories on the site automaticaly and better. Users would not use OpenCalais at all, it will make live of the neighbours easier. This could be used in views with exposed filters.

Quote:

Theme

The genesis theme you are using is not really a theme that is that easy to theme as compared with other more popular themes. At the same time, given the amount of resources invested in this site, why not to buy a theme for about 300Eur that could make it a lot easier to theme on a framework that is very developed. It is obvious right now that the theme is not looking very well and I also found some issues with code just on a very superficial look.

I am not sure I agree with you here. Of course, when you start using a new base theme, there is a learning curve. We choose Genisis because it is flexible to configure custom regions, and it has its own gpanels technology. The idea was not to use the panels module (more on this later).
About buying a drupal theme: you should first read this and another article.
The theme is not looking great at the moment: because theming is not finished yet.
Quote:

Using the power of popular modules

it would be great to give more power to modules like views that make it a lot easier for people to search content which is for example one of the things that are most important for people. Why not use more exposed filters to search content

No question about this. Use views, deploy views where-ever you can.
Quote:

Why is panels not being used. One of the most powerful modules at the moment.

We use panels in the current site, and it is a great module. There where a few reasons why we didn't want to use Panels in the new site:

* Panels was not available for Drupal 6 when we started developing the new site.
* Panels carries a performance hit because the layout variables live in the database and a lot of code is executed to render the pages.
* Caching of panels and panes works fine on a static site, but is much more difficult in a community site where content is added every minute.
* Genesis whith gpanels can do the trick.

On the other hand, because of the new situation, the finishing of the new site will be done by volunteer developers. That could be a reason to use panels again.

Some last remarks:

* There is a lot of exsisting data. About 180.000 nodes, 17.500 files, 20.000 embedded videos, 50.000 external links and much more. People have put a lot of effort to write and upload this content, don't let it get lost.
* There are a lot of long time functionalities and procedures like villages, projects, tasks, discussions, wiki's. The community has to decide it they want to loose any of these.
* Most of the groups are just used as forums. Investigate if converting the groups to forums would be feasable and make the site easier.

Migrating the current site to an easier to use new one is a big challenge.

Hope this clears up the questions to have about the new website for Nabuur.com.

0
  • 12 reads
stdClass Object ( [nid] => 256808 [vid] => 259208 [type] => task_discussion [status] => 1 [created] => 1261139715 [changed] => 1261139715 [comment] => 0 [promote] => 1 [sticky] => 0 [revision_timestamp] => 1261139715 [title] => Panels [body] => Moving to work with panels would have the bonus of being more democratic, in that anyone, even people without much experience of Drupal and no PHP could make, change and test layouts. The live versions can, and should, be dropped into code, so that's no problem. It's a fair weight of extra module (and markup) but maybe caching will help. Versioning, or merging between different peoples changes becomes a more manual task. However moving to panels now does have the added extra work of taking what has been done with drupal menu hook overrides and simple php templates into the panels. I think Roy will have some things to add onto the panels / gpanels discussion too... [teaser] => Moving to work with panels would have the bonus of being more democratic, in that anyone, even people without much experience of Drupal and no PHP could make, change and test layouts. The live versions can, and should, be dropped into code, so that's no problem. It's a fair weight of extra module (and markup) but maybe caching will help. Versioning, or merging between different peoples changes becomes a more manual task. However moving to panels now does have the added extra work of taking what has been done with drupal menu hook overrides and simple php templates into the panels. [log] => [format] => 1 [uid] => 17035 [name] => kester [picture] => [data] => a:9:{s:8:"og_email";s:1:"2";s:16:"privatemsg_allow";i:1;s:28:"privatemsg_setmessage_notify";i:1;s:17:"messaging_default";s:4:"mail";s:7:"contact";i:0;s:29:"taxonomy_image_disable_images";i:0;s:16:"nodecomment_sort";s:1:"1";s:8:"sms_user";a:1:{i:0;N;}s:14:"tinymce_status";s:4:"true";} [path] => group/nabuurcom-website-development/project/task/discussion/panels [field_embedded_video] => Array ( [0] => Array ( [embed] => [value] => [provider] => [data] => Array ( ) ) ) [field_external_link] => Array ( [0] => Array ( [url] => [title] => [attributes] => N; ) [1] => Array ( [url] => [title] => [attributes] => N; ) ) [comment_type] => task_discussion [og_groups] => Array ( [0] => 14315 ) [og_groups_both] => Array ( [14315] => NABUUR.com Website maintenance and development team ) [og_public] => 1 [tags] => [nodewords] => Array ( ) [trid] => 0 [language] => en [i18n_status] => 0 [taxonomy] => Array ( ) [translation] => Array ( ) [cid] => 256808 [pid] => 256807 [hostname] => 83.163.41.240 [thread] => 01.00/ [mail] => [homepage] => [comment_target_nid] => 256806 [depth] => 1 )
kester
Panels
Fri, 2009/12/18 - 14:35

Moving to work with panels would have the bonus of being more democratic, in that anyone, even people without much experience of Drupal and no PHP could make, change and test layouts. The live versions can, and should, be dropped into code, so that's no problem. It's a fair weight of extra module (and markup) but maybe caching will help. Versioning, or merging between different peoples changes becomes a more manual task.

However moving to panels now does have the added extra work of taking what has been done with drupal menu hook overrides and simple php templates into the panels.

I think Roy will have some things to add onto the panels / gpanels discussion too...

0
  • 19 reads
stdClass Object ( [nid] => 256811 [vid] => 259214 [type] => task_discussion [status] => 1 [created] => 1261140439 [changed] => 1261141041 [comment] => 0 [promote] => 1 [sticky] => 0 [revision_timestamp] => 1261141041 [title] => Solr [body] => Solr rocks. There's just no other way to describe it. If we can manage the infrastructure is a question of what gets offered. I might be able to help via-via there we shall see. The last thing solr is doing is adding weight. At it's most simple it's taking one of the greatest loads of the mysql server - search; and it's doing it better! But it can do more. A great deal of the nabuur platform is about finding the tasks and resources that are relevant, and using solr for this is ideal. It opens the way to future views3 integration as well! I'd embed it here but emfield D5 doesn't do archive.org:- http://www.archive.org/details/DrupalconDc2009-MoreThanSearchHowApachesolrChangesTheWayYouBuild [teaser] => Solr rocks. There's just no other way to describe it. If we can manage the infrastructure is a question of what gets offered. I might be able to help via-via there we shall see. The last thing solr is doing is adding weight. At it's most simple it's taking one of the greatest loads of the mysql server - search; and it's doing it better! But it can do more. A great deal of the nabuur platform is about finding the tasks and resources that are relevant, and using solr for this is ideal. It opens the way to future views3 integration as well! [log] => [format] => 1 [uid] => 17035 [name] => kester [picture] => [data] => a:9:{s:8:"og_email";s:1:"2";s:16:"privatemsg_allow";i:1;s:28:"privatemsg_setmessage_notify";i:1;s:17:"messaging_default";s:4:"mail";s:7:"contact";i:0;s:29:"taxonomy_image_disable_images";i:0;s:16:"nodecomment_sort";s:1:"1";s:8:"sms_user";a:1:{i:0;N;}s:14:"tinymce_status";s:4:"true";} [path] => group/nabuurcom-website-development/project/task/discussion/solr [field_embedded_video] => Array ( [0] => Array ( [embed] => [value] => [provider] => [data] => Array ( ) ) ) [field_external_link] => Array ( [0] => Array ( [url] => [title] => [attributes] => N; ) [1] => Array ( [url] => [title] => [attributes] => N; ) ) [comment_type] => task_discussion [og_groups] => Array ( [0] => 14315 ) [og_groups_both] => Array ( [14315] => NABUUR.com Website maintenance and development team ) [og_public] => 1 [tags] => [nodewords] => Array ( ) [trid] => 0 [language] => en [i18n_status] => 0 [taxonomy] => Array ( ) [translation] => Array ( ) [cid] => 256811 [pid] => 256807 [hostname] => 83.163.41.240 [thread] => 01.01/ [mail] => [homepage] => [comment_target_nid] => 256806 [depth] => 1 )
kester
Solr
Fri, 2009/12/18 - 14:47

Solr rocks. There's just no other way to describe it. If we can manage the infrastructure is a question of what gets offered. I might be able to help via-via there we shall see.

The last thing solr is doing is adding weight. At it's most simple it's taking one of the greatest loads of the mysql server - search; and it's doing it better! But it can do more. A great deal of the nabuur platform is about finding the tasks and resources that are relevant, and using solr for this is ideal.

It opens the way to future views3 integration as well!

I'd embed it here but emfield D5 doesn't do archive.org:- http://www.archive.org/details/DrupalconDc2009-MoreThanSearchHowApacheso...

0
  • 28 reads
stdClass Object ( [nid] => 256813 [vid] => 259213 [type] => task_discussion [status] => 1 [created] => 1261140957 [changed] => 1261140957 [comment] => 0 [promote] => 1 [sticky] => 0 [revision_timestamp] => 1261140957 [title] => Calais [body] => I guess this is a take it or leave it decision. Really it costs the site a little bit of code weight - it requires RDF too. There's no additional bandwidth to the user, and a small one-off output and download bandwidth per posting. The benefits could well be worth the effort though. There's only one way of finding out - using it! Adding semantic analysis for tagging could well make all the finding of content that's so difficult at the moment much easier. Freetagging, even choosing from preset terms, at the moment just doesn't work well. Advertising the data in standardised RDF format, and having standard tags also isn't going to do any harm for promoting the site. I'd say lets test it and see, if code weight is an issue on the server, then it can be one of the first to go. [teaser] => I guess this is a take it or leave it decision. Really it costs the site a little bit of code weight - it requires RDF too. There's no additional bandwidth to the user, and a small one-off output and download bandwidth per posting. The benefits could well be worth the effort though. There's only one way of finding out - using it! [log] => [format] => 1 [uid] => 17035 [name] => kester [picture] => [data] => a:9:{s:8:"og_email";s:1:"2";s:16:"privatemsg_allow";i:1;s:28:"privatemsg_setmessage_notify";i:1;s:17:"messaging_default";s:4:"mail";s:7:"contact";i:0;s:29:"taxonomy_image_disable_images";i:0;s:16:"nodecomment_sort";s:1:"1";s:8:"sms_user";a:1:{i:0;N;}s:14:"tinymce_status";s:4:"true";} [path] => group/nabuurcom-website-development/project/task/discussion/calais [field_embedded_video] => Array ( [0] => Array ( [embed] => [value] => [provider] => [data] => Array ( ) ) ) [field_external_link] => Array ( [0] => Array ( [url] => [title] => [attributes] => N; ) [1] => Array ( [url] => [title] => [attributes] => N; ) ) [comment_type] => task_discussion [og_groups] => Array ( [0] => 14315 ) [og_groups_both] => Array ( [14315] => NABUUR.com Website maintenance and development team ) [og_public] => 1 [tags] => [nodewords] => Array ( ) [trid] => 0 [language] => en [i18n_status] => 0 [taxonomy] => Array ( ) [translation] => Array ( ) [cid] => 256813 [pid] => 256807 [hostname] => 83.163.41.240 [thread] => 01.02/ [mail] => [homepage] => [comment_target_nid] => 256806 [depth] => 1 )
kester
Calais
Fri, 2009/12/18 - 14:55

I guess this is a take it or leave it decision. Really it costs the site a little bit of code weight - it requires RDF too. There's no additional bandwidth to the user, and a small one-off output and download bandwidth per posting.

The benefits could well be worth the effort though. There's only one way of finding out - using it!

Adding semantic analysis for tagging could well make all the finding of content that's so difficult at the moment much easier. Freetagging, even choosing from preset terms, at the moment just doesn't work well. Advertising the data in standardised RDF format, and having standard tags also isn't going to do any harm for promoting the site.

I'd say lets test it and see, if code weight is an issue on the server, then it can be one of the first to go.

0
  • 21 reads