L-electron-Rare/moodle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Big global search update for 1.9. Will resolve major bug points in Global Search component section.

Browse Source
This commit is contained in:
diml
2008-12-17 20:51:19 +00:00
parent 68c5857cb9
commit 323a68467c
15 changed files with 782 additions and 562 deletions
Download Patch File Download Diff File Expand all files Collapse all files
search/LISEZMOI.txt
+40 -40
View File
@@ -2,59 +2,56 @@ Cette distribution partielle contient une refonte du moteur de
recherche globale de Moodle.
Le moteur de recherche est capable d'indexer et de rechercher
des informations dans un grand nombre de contenus stockés
dans la plate-forme à travers la manipulation des activités et
des informations dans un grand nombre de contenus stockés
dans la plate-forme à travers la manipulation des activités et
des blocs.
Le moteur de recherche procède à une première indexation des
Le moteur de recherche procède à une première indexation des
ressources disponibles par action de l'administrateur. Une fois
cette indexation effectuée, le moteur maintient régulièrement les
indexes, en ajoutant les nouvelles entrées et en nettoyant les
entrées obsolètes.
cette indexation effectuée, le moteur maintient régulièrement les
indexes, en ajoutant les nouvelles entrées et en nettoyant les
entrées obsolètes.
La recherche permet d'obtenir des références d'accès au contexte
La recherche permet d'obtenir des références d'accès au contexte
qui diffuse cette information, au nom de l'utilisateur courant.
Le filtrage des résultats enlève de la liste des réponses toute
ressource que la situation de l'utilisateur empêcherait de voir
s'il y accédait dans son contexte habituel.
Le filtrage des résultats enlève de la liste des réponses toute
ressource que la situation de l'utilisateur empêcherait de voir
s'il y accédait dans son contexte habituel.
Mise en oeuvre
##############
Pour déployer le moteur :
La distribution fait désormais partie du noyau de Moodle.
Il sera probablement nécessaire d'ajouter un certain nombre de librairies additionnelles
pour la conversion de documents physiques en vue de leur indexation. Ces librairies sont
actuellement fournies dans le CVS dans la rubrique contrib/patches/global_search_libraries
(antiword et xpdf). La prise en charge des fichiers "shockwave" est assurée, sous réserve
de l'obtention des libairies de conversion auprès de Adobe (http://www.adobe.com/licensing/developer/)
* Copie de fichiers
1. Allez sur le bloc d'administration et réglez les paramètres du bloc Recherche Globale.
Ceci initialisera un certain nombre de fonctions dans le moteur.
1. Ajouter les deux librairies fournies aux librairies de Moodle
2. Ecraser le répertoire "search" par le répertoire fourni
3. Ecraser le bloc "blocs/search" par le bloc fourni.
2. Insérer un nouveau bloc de recherche globale dans la plate-forme
* Installation logique
3. Effectuer une recherche vide (en administrateur)
4. Aller dans les notifications administratives et dérouler la procédure d'installation/mise à jour du bloc. L'installation crée la table image
des documents indexés et utilisés dans le module search.
4. Aller sur la page des statistiques
5. Insérer un nouveau bloc de recherche globale dans la plate-forme
5. Activer l'indexation (indexsplash.php). Attention, si la plate-form contient beaucoup de contenus cette indexation peut être TRES LONGUE.
6. Effectuer une recherche vide (en administrateur)
Pour effectuer des recherches, une fois la première indexation terminée, retourner au bloc de recherche et tenter une recherche.
7. Aller sur la page des statistiques
8. Activer l'indexation (indexsplash.php). Attention, si la plate-form contient beaucoup de contenus cette indexation peut être TRES LONGUE.
Pour effectuer des recherches, une fois la première indexation terminée, retourner au bloc de recherche et tenter une recherche.
Eléments pris en charge
Eléments pris en charge
#######################
Dans l'état actuel, les éléments indexés par le moteur sont :
Dans l'état actuel, les éléments indexés par le moteur sont :
- les entrées de forum
- les fiches de base de données
- les commentaires sur fiches de données
- les entrées de glossaire
- les commentaires sur entrées de glossaire
- les entrées de forum
- les fiches de base de données
- les commentaires sur fiches de données
- les entrées de glossaire
- les commentaires sur entrées de glossaire
- les ressources natives Moodle
- les ressources physiques de type MSWord
- les ressources physiques de type PDF
@@ -63,27 +60,30 @@ Dans l'état actuel, les éléments indexés par le moteur sont :
- les ressources physiques de type XML (.xml)
- les ressources physiques de type (Microsoft) Powerpoint (.ppt)
- les pages de wiki
- les entités de projet technique
- les sessions de chat
Des modules tiers ont été rendus indexables
- Techproject
Extensions
##########
L'API du moteur de recherche permet désormais :
L'API du moteur de recherche permet désormais :
- l'indexation de contenus de blocs.
- l'indexation de modules contenant une information complexe ou de plusieurs types distincts
- la sécurisation des informations indexées lors des extractions de résultats
- l'indexation de tout module tiers par ajout d'un fichier php calibré
- l'indexation de toute nouvelle resource physique par ajout d'un fichier php calibré
- la sécurisation des informations indexées lors des extractions de résultats
- l'indexation de tout module tiers par ajout d'un fichier php calibré
- l'indexation de toute nouvelle resource physique par ajout d'un fichier php calibré
Extensions futures
##################
- De nouvelles prises en charge de contenus tels que les attachements des forums, les attachement des glossaires, ainsi que d'autres modules non encore
implémentés.
implémentés.
- l'extension mnet de la recherche dans un réseau de moodle interconnectés.
- l'extension mnet de la recherche dans un réseau de moodle interconnectés.
search/README.txt
+68 -129
View File
@@ -1,153 +1,92 @@
2006/09/08
----------
Google Summer of Code is finished, spent a couple of weeks away from
the project to think about it and also to take a break. Working on it
now I discovered bugs in the query parser (now fixed), and I also
un-convoluted the querylib logic (well slighlty).
This directoery contains the central implementation of
Moodle's Global Search Engine.
Updated ZFS files to latest SVN.
The Global Search Engine stores indexes about a huge quantity
of information from within modules, block or resources stored
by Moodle either in the database or the file system.
2006/08/21
----------
Fixed index document count, and created new config variable to store
the size. (Search now has 3 global vars in $CFG, date, size and complete,
see indexer.php for var names). Index size is cached to provide an always
current value for the index - this is to take into account the fact that
deleted documents are in fact not removed from the index, but instead just
marked as deleted and not returned in search results. The actual document
still features in the index, and skews sizes. When the index optimiser is
completed in ZFS, then these deleted documents will be pruned, thus
correctly modifying the index size.
The administrator initialy indexes the existing content. Once this
first initialization performed, the search engine maintains indexes
regularily, adding new entries, deleting obsolete one or updating
some that have changed.
Additional commenting added.
Search will produce links for acceding the information in a similar
context as usually accessed, from the current user point of view.
Results filtering removes from results any link to information the
current user would not be allowed to acces on a straight situation.
Query page logic very slightly modified to clean up GET string a bit (removed
'p' variable).
Deployment
###########
Add/delete functions added to other document types.
The search engine is now part of Moodle core distribution.
A few TODO fields added to source, indicating changes still to come (or at
least to be considered).
Some extra libraries might be added for converting physical documents to text
so it can be indexed. Moodle CVS (entry contrib/patches/global_search_libraries)
provides packs for antiword and xpdf GPL libraries the search engine is ready for
shockwave indexing, but will not provide Adobe Search converters that should be
obtained at http://www.adobe.com/licensing/developer/
2006/08/16
----------
Add/delete/update cron functions finished - can be called seperately
or all at once via cron.php.
1. Go to the block administration panel and setup once the Global Search
block. This will initialize useful parameters for the global search engine.
Document date field added to index and database summary.
2. Insert a new Global Search block somewhere in a course or top-level screen.
Some index db functionality abstracted out to indexlib.php - can
use IndexDBControl class to add/del documents from database, and
to make sure the db table is functioning.
3. Launch an empty search (you must be administrator).
DB sql files changed to add some extra fields.
4. Go to the statistics screen.
Default 'simple' query modified to search title and author, as well
as contents of document, to provide better results for users.
5. Activate indexation (indexersplash.php). Beware, if your Moodle has
a large amount of content, indexing process may be VERY LONG.
2006/08/14
----------
First revision of the advanced search page completed. Functional,
but needs a date search field still.
To search, go back to the search block and try a query.
2006/08/02
----------
Added resource search type, and the ability to specify custom 'virtual'
models to search - allowing for non-module specific information to be
indexed. Specify the extra search types to use in lib.php.
Handled information for indexing
################################
2006/07/28
----------
Added delete logic to documents; the moodle database log is checked
and any found delete events are used to remove the referenced documents
from the database table and search index.
In the actual state, the engine indexes the following information:
Added database table name constant to lib.php, must change files using
the static table name.
- assignment descriptions
- forum posts
- database records (using textual fields only)
- database comments
- glossary entries
- glossary comments on entries
- Moodle native resources
- physical MSWord files as resources (.doc)
- physical Powerpoint files as resources (.ppt)
- physical PDF files as resources
- physical text files as resources (.txt)
- physical html files as resources (.htm and .html)
- physical xml files as resources (.xml)
- wiki pages
- chat sessions
- lesson pages
Changed documents to use 'docid' instead of 'id' to reference the moodle
instance id, since Zend Search adds it's own internal 'id' field. Noticed
this whilst working on deletions.
Some third party plugins are also searchable using the new Search API implementation
Added some additional fields to the permissions checking method, must still
implement it though.
- Techproject
2006/07/25
----------
Query logic moved into the SearchQuery class in querylib.php. Should be able
to include this file in any page and run a query against the index (PHP 5
checks must be added to those pages then, though).
Extensions
##########
Index info can be retrieved using IndexInfo class in indexlib.php.
The reviewed search engine API allows:
Abstracted some stuff away, to reduce rendundancy and decrease the
likelihood of errors. Improved the stats.php page to include some
diagnostics for adminstrators.
- indexing of blocks contents
- indexation of modules or blocks containing a complex information model
- securing the access to the results
- adding indexing handling for additional modules and plugins adding a php calibrated script
- adding physical filetype handling adding a php calibrated script
delete.php skeleton created for removing deleted documents from the
index. cron.php will contain the logic for running delete.php,
update.php and eventually add.php.
Global Search on NFS Mounted clusters
#####################################
2006/07/11
----------
(Warning: It took me 1900 seconds to index the forum, go make coffee
whilst you wait.) [Moodle.org forum data]
This version contains a patched Lucene Zend implementation that allows using the Global Search engine in an NFS mounted shared volume for Web clustering. This implementation
remains highly experimental and not all tests have been processed. Some changes may
occur in the SoftLockManager that was added to the Lucene engine.
Forum search functions changed to use 'get_recordset' instead of
'get_records', for speed reasons. This provides a significant improvement,
but indexing is still slow - getting data from the database and Zend's
tokenising _seem_ to be the prime suspects at the moment.
Future extensions
#################
/search/tests/ added - index.php can be used to see which modules are
ready to be included in the search index, and it informs you of any
errors - should be a prerequisite for indexing.
Search result pagination added to query.php, will default to 20 until
an admin page for the search module is written.
2006/07/07
----------
Search-enabling functions moved out've the mod's lib.php files and into
/search/documents/mod_document.php - this requires the search module to
operate without requiring modification of lib files.
SearchDocument base class improved, and the way module documents extend
it. A custom-data field has been added to allow modules to add any custom
data they wish to be stored in the index - this field is serialised into
the index as a binary field.
Database field 'type' renamed to 'doctype' to match the renaming in the
index, 'type' seems to be a reserved word in Lucene. Several index field
names change to be more descriptive (cid -> course_id). URLs are now
stored in the index, and don't have to be generated on the fly during
display of query results.
2006/07/05
------
Started cleaning and standardising things.
cvs v1.1
--------
This is the initial release (prototype) of Moodle's new search module -
so basically watch out for sharp edges.
The structure has not been finalised, but this is what is working at the
moment, when I start looking at other content to index, it will most likely
change. I don't recommend trying to make your own content modules indexable,
at least not until the whole flow is finalised. I will be implementing the
functions needed to index all of the default content modules on Moodle, so
expect that around mid-August.
Wiki pages were my goal for this release, they can be indexed and searched,
but not updated or deleted at this stage (was waiting for ZF 0.14 actually).
I need to check the PostgreSQL sql file, I don't have a PG7 install lying
around to test on, so the script is untested.
To index for the first time, login as an admin user and browse to /search/index.php
or /search/stats.php - there will be a message and a link telling you to go index.
-- Michael Champanis (mchampan)
email: [email protected]
skype: mchampan
Summer of Code 2006
- Should be added more information to index such as forum and glossary attachements,
so will other standard module contents.
- extending the search capability to a mnet network information space by aggregating remote search responses.
search/README_ARCHIVE.txt
+153
View File
@@ -0,0 +1,153 @@
2006/09/08
----------
Google Summer of Code is finished, spent a couple of weeks away from
the project to think about it and also to take a break. Working on it
now I discovered bugs in the query parser (now fixed), and I also
un-convoluted the querylib logic (well slighlty).
Updated ZFS files to latest SVN.
2006/08/21
----------
Fixed index document count, and created new config variable to store
the size. (Search now has 3 global vars in $CFG, date, size and complete,
see indexer.php for var names). Index size is cached to provide an always
current value for the index - this is to take into account the fact that
deleted documents are in fact not removed from the index, but instead just
marked as deleted and not returned in search results. The actual document
still features in the index, and skews sizes. When the index optimiser is
completed in ZFS, then these deleted documents will be pruned, thus
correctly modifying the index size.
Additional commenting added.
Query page logic very slightly modified to clean up GET string a bit (removed
'p' variable).
Add/delete functions added to other document types.
A few TODO fields added to source, indicating changes still to come (or at
least to be considered).
2006/08/16
----------
Add/delete/update cron functions finished - can be called seperately
or all at once via cron.php.
Document date field added to index and database summary.
Some index db functionality abstracted out to indexlib.php - can
use IndexDBControl class to add/del documents from database, and
to make sure the db table is functioning.
DB sql files changed to add some extra fields.
Default 'simple' query modified to search title and author, as well
as contents of document, to provide better results for users.
2006/08/14
----------
First revision of the advanced search page completed. Functional,
but needs a date search field still.
2006/08/02
----------
Added resource search type, and the ability to specify custom 'virtual'
models to search - allowing for non-module specific information to be
indexed. Specify the extra search types to use in lib.php.
2006/07/28
----------
Added delete logic to documents; the moodle database log is checked
and any found delete events are used to remove the referenced documents
from the database table and search index.
Added database table name constant to lib.php, must change files using
the static table name.
Changed documents to use 'docid' instead of 'id' to reference the moodle
instance id, since Zend Search adds it's own internal 'id' field. Noticed
this whilst working on deletions.
Added some additional fields to the permissions checking method, must still
implement it though.
2006/07/25
----------
Query logic moved into the SearchQuery class in querylib.php. Should be able
to include this file in any page and run a query against the index (PHP 5
checks must be added to those pages then, though).
Index info can be retrieved using IndexInfo class in indexlib.php.
Abstracted some stuff away, to reduce rendundancy and decrease the
likelihood of errors. Improved the stats.php page to include some
diagnostics for adminstrators.
delete.php skeleton created for removing deleted documents from the
index. cron.php will contain the logic for running delete.php,
update.php and eventually add.php.
2006/07/11
----------
(Warning: It took me 1900 seconds to index the forum, go make coffee
whilst you wait.) [Moodle.org forum data]
Forum search functions changed to use 'get_recordset' instead of
'get_records', for speed reasons. This provides a significant improvement,
but indexing is still slow - getting data from the database and Zend's
tokenising _seem_ to be the prime suspects at the moment.
/search/tests/ added - index.php can be used to see which modules are
ready to be included in the search index, and it informs you of any
errors - should be a prerequisite for indexing.
Search result pagination added to query.php, will default to 20 until
an admin page for the search module is written.
2006/07/07
----------
Search-enabling functions moved out've the mod's lib.php files and into
/search/documents/mod_document.php - this requires the search module to
operate without requiring modification of lib files.
SearchDocument base class improved, and the way module documents extend
it. A custom-data field has been added to allow modules to add any custom
data they wish to be stored in the index - this field is serialised into
the index as a binary field.
Database field 'type' renamed to 'doctype' to match the renaming in the
index, 'type' seems to be a reserved word in Lucene. Several index field
names change to be more descriptive (cid -> course_id). URLs are now
stored in the index, and don't have to be generated on the fly during
display of query results.
2006/07/05
------
Started cleaning and standardising things.
cvs v1.1
--------
This is the initial release (prototype) of Moodle's new search module -
so basically watch out for sharp edges.
The structure has not been finalised, but this is what is working at the
moment, when I start looking at other content to index, it will most likely
change. I don't recommend trying to make your own content modules indexable,
at least not until the whole flow is finalised. I will be implementing the
functions needed to index all of the default content modules on Moodle, so
expect that around mid-August.
Wiki pages were my goal for this release, they can be indexed and searched,
but not updated or deleted at this stage (was waiting for ZF 0.14 actually).
I need to check the PostgreSQL sql file, I don't have a PG7 install lying
around to test on, so the script is untested.
To index for the first time, login as an admin user and browse to /search/index.php
or /search/stats.php - there will be a message and a link telling you to go index.
-- Michael Champanis (mchampan)
email: [email protected]
skype: mchampan
Summer of Code 2006
search/add.php
+38 -35
View File
@@ -1,25 +1,31 @@
<?php
/**
* Global Search Engine for Moodle
*
* @package search
* @category core
* @subpackage search_engine
* @author Michael Champanis (mchampan) [[email protected]], Valery Fremaux [[email protected]] > 1.8
* @date 2008/03/31
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*
* Asynchronous adder for new indexable contents
*
* Major chages in this review is passing the xxxx_db_names return to
* multiple arity to handle multiple document types modules
*/
/**
* Global Search Engine for Moodle
*
* @package search
* @category core
* @subpackage search_engine
* @author Michael Champanis (mchampan) [[email protected]], Valery Fremaux [[email protected]] > 1.8
* @date 2008/03/31
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*
* Asynchronous adder for new indexable contents
*
* Major chages in this review is passing the xxxx_db_names return to
* multiple arity to handle multiple document types modules
*/
/**
* includes and requires
*/
require_once('../config.php');
/**
* includes and requires
*/
require_once('../config.php');
require_once("$CFG->dirroot/search/lib.php");
/// makes inclusions of the Zend Engine more reliable
$separator = (array_key_exists('WINDIR', $_SERVER)) ? ';' : ':' ;
ini_set('include_path', $CFG->dirroot.'/search'.$separator.ini_get('include_path'));
require_once("$CFG->dirroot/search/lib.php");
require_once("$CFG->dirroot/search/indexlib.php");
/// checks global search activation
@@ -29,21 +35,18 @@ require_once("$CFG->dirroot/search/lib.php");
error(get_string('globalsearchdisabled', 'search'));
}
if (!isadmin()) {
if (!has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))) {
error(get_string('beadmin', 'search'), "$CFG->wwwroot/login/index.php");
}
/// check for php5 (lib.php)
if (!search_check_php5()) {
$phpversion = phpversion();
mtrace("Sorry, global search requires PHP 5.0.0 or later (currently using version ".phpversion().")");
exit(0);
}
require_once("$CFG->dirroot/search/indexlib.php");
$index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
try {
$index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
} catch(LuceneException $e) {
mtrace("Could not construct a valid index. Maybe the first indexation was never made, or files might be corrupted. Run complete indexation again.");
return;
}
$dbcontrol = new IndexDBControl();
$addition_count = 0;
$startindextime = time();
@@ -76,7 +79,7 @@ require_once("$CFG->dirroot/search/lib.php");
$valuesArray = $db_names_function();
if ($valuesArray){
foreach($valuesArray as $values){
$where = (isset($values[5])) ? 'AND ('.$values[5].')' : '';
$where = (!empty($values[5])) ? 'AND ('.$values[5].')' : '';
$itemtypes = ($values[4] != '*' && $values[4] != 'any') ? " AND itemtype = '{$values[4]}' " : '' ;
//select records in MODULE table, but not in SEARCH_DATABASE_TABLE
@@ -88,10 +91,10 @@ require_once("$CFG->dirroot/search/lib.php");
FROM
{$CFG->prefix}{$table}
WHERE
doctype = '{$mod->name}'
doctype = '{$values[1]}'
$itemtypes
";
$docIds = get_records_sql_menu($query);
$docIds = get_records_sql_menu($query, array($mod->name));
$docIdList = ($docIds) ? implode("','", array_keys($docIds)) : '' ;
$query = "
@@ -100,7 +103,7 @@ require_once("$CFG->dirroot/search/lib.php");
FROM
{$CFG->prefix}{$values[1]}
WHERE
id NOT IN ('{$docIdList}') and
id NOT IN ('{$docIdList}') AND
{$values[2]} > {$indexdate}
$where
";
@@ -155,4 +158,4 @@ require_once("$CFG->dirroot/search/lib.php");
mtrace("Added $addition_count documents.");
mtrace('Index size after: '.$index->count());
?>
?>
search/cron.php
+1 -5
View File
@@ -18,11 +18,7 @@
require_once("$CFG->dirroot/search/lib.php");
if (!search_check_php5()) {
$phpversion = phpversion();
mtrace("Sorry, cannot cron global search as it requires PHP 5.0.0 or later (currently using version $phpversion)");
}
else if (empty($CFG->enableglobalsearch)) {
if (empty($CFG->enableglobalsearch)) {
mtrace('Global searching is not enabled. Nothing performed by search.');
}
else{
search/delete.php
+38 -33
View File
@@ -1,47 +1,51 @@
<?php
/**
* Global Search Engine for Moodle
*
* @package search
* @category core
* @subpackage search_engine
* @author Michael Champanis (mchampan) [[email protected]], Valery Fremaux [[email protected]] > 1.8
* @date 2008/03/31
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*
* Asynchronous index cleaner
*
* Major chages in this review is passing the xxxx_db_names return to
* multiple arity to handle multiple document types modules
*/
/**
* Global Search Engine for Moodle
*
* @package search
* @category core
* @subpackage search_engine
* @author Michael Champanis (mchampan) [[email protected]], Valery Fremaux [[email protected]] > 1.8
* @date 2008/03/31
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*
* Asynchronous index cleaner
*
* Major chages in this review is passing the xxxx_db_names return to
* multiple arity to handle multiple document types modules
*/
/**
* includes and requires
*/
require_once('../config.php');
/**
* includes and requires
*/
require_once('../config.php');
require_once("$CFG->dirroot/search/lib.php");
/// makes inclusions of the Zend Engine more reliable
$separator = (array_key_exists('WINDIR', $_SERVER)) ? ';' : ':' ;
ini_set('include_path', $CFG->dirroot.'/search'.$separator.ini_get('include_path'));
require_once("$CFG->dirroot/search/lib.php");
require_once("$CFG->dirroot/search/indexlib.php");
/// checks global search activation
require_login();
if (empty($CFG->enableglobalsearch)) {
error(get_string('globalsearchdisabled', 'search'));
}
if (!isadmin()) {
if (!has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))) {
error(get_string('beadmin', 'search'), "$CFG->wwwroot/login/index.php");
} //if
//check for php5 (lib.php)
if (!search_check_php5()) {
$phpversion = phpversion();
mtrace("Sorry, global search requires PHP 5.0.0 or later (currently using version ".phpversion().")");
exit(0);
try {
$index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
} catch(LuceneException $e) {
mtrace("Could not construct a valid index. Maybe the first indexation was never made, or files might be corrupted. Run complete indexation again.");
return;
}
require_once("$CFG->dirroot/search/indexlib.php");
$index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
$dbcontrol = new IndexDBControl();
$deletion_count = 0;
$startcleantime = time();
@@ -49,6 +53,7 @@ require_once("$CFG->dirroot/search/lib.php");
mtrace('Starting clean-up of removed records...');
mtrace('Index size before: '.$CFG->search_index_size."\n");
/// check all modules
if ($mods = get_records_select('modules')) {
$mods = array_merge($mods, search_get_additional_modules());
@@ -68,7 +73,7 @@ require_once("$CFG->dirroot/search/lib.php");
$valuesArray = $db_names_function();
if ($valuesArray){
foreach($valuesArray as $values){
$where = (isset($values[5])) ? 'WHERE '.$values[5] : '';
$where = (!empty($values[5])) ? 'WHERE '.$values[5] : '';
$itemtypes = ($values[4] != '*' && $values[4] != 'any') ? " itemtype = '{$values[4]}' AND " : '' ;
$query = "
SELECT
@@ -139,4 +144,4 @@ require_once("$CFG->dirroot/search/lib.php");
mtrace("Finished $deletion_count removals.");
mtrace('Index size after: '.$index->count());
?>
?>
search/indexer.php
+42 -31
View File
@@ -35,6 +35,10 @@
*/
require_once('../config.php');
require_once("$CFG->dirroot/search/lib.php");
//require_once("debugging.php");
$separator = (array_key_exists('WINDIR', $_SERVER)) ? ';' : ':' ;
ini_set('include_path', $CFG->dirroot.'\search'.$separator.ini_get('include_path'));
/// only administrators can index the moodle installation, because access to all pages is required
@@ -44,7 +48,7 @@ require_once("$CFG->dirroot/search/lib.php");
error(get_string('globalsearchdisabled', 'search'));
}
if (!isadmin()) {
if (!has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))) {
error(get_string('beadmin', 'search'), "$CFG->wwwroot/login/index.php");
}
@@ -61,12 +65,6 @@ require_once("$CFG->dirroot/search/lib.php");
/// check for php5 (lib.php)
if (!search_check_php5()) {
$phpversion = phpversion();
mtrace("Sorry, global search requires PHP 5.0.0 or later (currently using version ".phpversion().")");
exit(0);
}
//php5 found, continue including php5-only files
//require_once("$CFG->dirroot/search/Zend/Search/Lucene.php");
require_once("$CFG->dirroot/search/indexlib.php");
@@ -94,13 +92,16 @@ require_once("$CFG->dirroot/search/lib.php");
mtrace("Data directory ($index_path) does not exist, attempting to create.");
if (!mkdir($index_path)) {
search_pexit("Error creating data directory at: $index_path. Please correct.");
} else {
}
else {
mtrace("Directory successfully created.");
}
} else {
mtrace("Using $index_path as data directory.");
}
else {
mtrace("Using {$index_path} as data directory.");
}
Zend_Search_Lucene_Analysis_Analyzer::setDefault(new Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8_CaseInsensitive());
$index = new Zend_Search_Lucene($index_path, true);
/*
@@ -124,31 +125,39 @@ require_once("$CFG->dirroot/search/lib.php");
// * mod_iterator
// * mod_get_content_for_index
//are the sole basis for including a module in the index at the moment.
$searchables = array();
$searchables = search_collect_searchables();
/// collects modules
/// start indexation
if ($mods = get_records('modules', '', '', '', 'id,name')) {
$searchables = array_merge($searchables, $mods);
}
mtrace(count($searchables).' modules found.');
// collects blocks as indexable information may be found in blocks either
if ($blocks = get_records('block', '', '', '', 'id,name')) {
// prepend the "block_" prefix to discriminate document type plugins
foreach(array_keys($blocks) as $aBlockId){
$blocks[$aBlockId]->name = 'block_'.$blocks[$aBlockId]->name;
}
$searchables = array_merge($searchables, $blocks);
mtrace(count($blocks).' blocks found.');
}
/// add virtual modules onto the back of the array
$searchables = array_merge($searchables, search_get_additional_modules());
if ($searchables){
foreach ($searchables as $mod) {
$class_file = $CFG->dirroot.'/search/documents/'.$mod->name.'_document.php';
echo "start {$mod->name}";
$key = 'search_in_'.$mod->name;
if (isset($CFG->$key) && !$CFG->$key) {
mtrace("module $key has been administratively disabled. Skipping...\n");
continue;
}
if ($mod->location == 'internal'){
$class_file = $CFG->dirroot.'/search/documents/'.$mod->name.'_document.php';
} else {
$class_file = $CFG->dirroot.'/'.$mod->location.'/'.$mod->name.'/search_document.php';
}
/*
if (!file_exists($class_file)){
if (defined("PATH_FOR_SEARCH_TYPE_{$mod->name}")){
eval("\$pluginpath = PATH_FOR_SEARCH_TYPE_{$mod->name}");
$class_file = "{$CFG->dirroot}/{$pluginpath}/searchlib.php";
} else {
mtrace ("No search document found for plugin {$mod->name}. Ignoring.");
continue;
}
}
*/
if (file_exists($class_file)) {
include_once($class_file);
@@ -195,6 +204,8 @@ require_once("$CFG->dirroot/search/lib.php");
mtrace("-- $counter documents indexed");
mtrace("done.\n");
}
} else {
mtrace ("No search document found for plugin {$mod->name}. Ignoring.");
}
}
}
search/indexersplash.php
+6 -10
View File
@@ -18,7 +18,11 @@
* includes and requires
*/
require_once('../config.php');
require_once("$CFG->dirroot/search/lib.php");
require_once("{$CFG->dirroot}/search/lib.php");
/// makes inclusions of the Zend Engine more reliable
$separator = (array_key_exists('WINDIR', $_SERVER)) ? ';' : ':' ;
ini_set('include_path', $CFG->dirroot.'\search'.$separator.ini_get('include_path'));
/// check global search is enabled
@@ -28,18 +32,10 @@ require_once("$CFG->dirroot/search/lib.php");
error(get_string('globalsearchdisabled', 'search'));
}
if (!isadmin()) {
if (!has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))) {
error(get_string('beadmin', 'search'), "$CFG->wwwroot/login/index.php");
}
/// check for php5 (lib.php)
if (!search_check_php5()) {
$phpversion = phpversion();
mtrace("Sorry, global search requires PHP 5.0.0 or later (currently using version ".phpversion().")");
exit(0);
}
require_once("$CFG->dirroot/search/indexlib.php");
$indexinfo = new IndexInfo();
search/indexlib.php
+2 -1
View File
@@ -78,7 +78,8 @@ class IndexInfo {
$this->dbcount = count_records(SEARCH_DATABASE_TABLE);
//individual document types
$types = search_get_document_types();
// $types = search_get_document_types();
$types = search_collect_searchables(true, false);
sort($types);
foreach($types as $type) {
search/lib.php
+85 -49
View File
@@ -16,41 +16,88 @@
*
*/
/*
// function reference
function search_get_document_types($prefix = 'SEARCH_TYPE_') {
function search_get_additional_modules() {
function search_shorten_url($url, $length=30) {
function search_escape_string($str) {
function search_check_php5($feedback = false) {
function search_stopwatch($cli = false) {
function search_pexit($str = "") {
/**
* Constants
*/
define('SEARCH_INDEX_PATH', "$CFG->dataroot/search");
define('SEARCH_INDEX_PATH', "{$CFG->dataroot}/search");
define('SEARCH_DATABASE_TABLE', 'block_search_documents');
//document types that can be searched
//define('SEARCH_TYPE_NONE', 'none');
define('SEARCH_TYPE_WIKI', 'wiki');
define('PATH_FOR_SEARCH_TYPE_WIKI', 'mod/wiki');
define('SEARCH_TYPE_FORUM', 'forum');
define('PATH_FOR_SEARCH_TYPE_FORUM', 'mod/forum');
define('SEARCH_TYPE_GLOSSARY', 'glossary');
define('PATH_FOR_SEARCH_TYPE_GLOSSARY', 'mod/glossary');
define('SEARCH_TYPE_RESOURCE', 'resource');
define('PATH_FOR_SEARCH_TYPE_RESOURCE', 'mod/resource');
define('SEARCH_TYPE_TECHPROJECT', 'techproject');
define('PATH_FOR_SEARCH_TYPE_TECHPROJECT', 'mod/techproject');
define('SEARCH_TYPE_DATA', 'data');
define('PATH_FOR_SEARCH_TYPE_DATA', 'mod/data');
define('SEARCH_TYPE_CHAT', 'chat');
define('PATH_FOR_SEARCH_TYPE_CHAT', 'mod/chat');
define('SEARCH_TYPE_LESSON', 'lesson');
define('PATH_FOR_SEARCH_TYPE_LESSON', 'mod/lesson');
// get document types
include "{$CFG->dirroot}/search/searchtypes.php";
/**
* returns all the document type constants
* collects all searchable items identities
* @param boolean $namelist if true, only returns list of names of searchable items
* @param boolean $verbose if true, prints a discovering status
* @return an array of names or an array of type descriptors
*/
function search_collect_searchables($namelist=false, $verbose=true){
global $CFG;
$searchables = array();
$searchables_names = array();
/// get all installed modules
if ($mods = get_records('modules', '', '', 'name', 'id,name')){
$searchabletypes = array_values(search_get_document_types());
foreach($mods as $mod){
if (in_array($mod->name, $searchabletypes)){
$mod->location = 'internal';
$searchables[$mod->name] = $mod;
$searchables_names[] = $mod->name;
} else {
$documentfile = $CFG->dirroot."/mod/{$mod->name}/search_document.php";
$mod->location = 'mod';
if (file_exists($documentfile)){
$searchables[$mod->name] = $mod;
$searchables_names[] = $mod->name;
}
}
}
if ($verbose) mtrace(count($searchables).' modules to search in / '.count($mods).' modules found.');
}
/// collects blocks as indexable information may be found in blocks either
if ($blocks = get_records('block', '', '', 'name', 'id,name')) {
$blocks_searchables = array();
// prepend the "block_" prefix to discriminate document type plugins
foreach($blocks as $block){
$block->dirname = $block->name;
$block->name = 'block_'.$block->name;
if (in_array('SEARCH_TYPE_'.strtoupper($block->name), $searchabletypes)){
$mod->location = 'internal';
$blocks_searchables[] = $block;
$searchables_names[] = $block->name;
} else {
$documentfile = $CFG->dirroot."/blocks/{$block->dirname}/search_document.php";
if (file_exists($documentfile)){
$mod->location = 'blocks';
$blocks_searchables[$block->name] = $block;
$searchables_names[] = $block->name;
}
}
}
if ($verbose) mtrace(count($blocks_searchables).' blocks to search in / '.count($blocks).' blocks found.');
$searchables = array_merge($searchables, $blocks_searchables);
}
/// add virtual modules onto the back of the array
$additional = search_get_additional_modules();
if (!empty($additional)){
if ($verbose) mtrace(count($additional).' additional to search in.');
$searchables = array_merge($searchables, $additional);
}
if ($namelist)
return $searchables_names;
return $searchables;
}
/**
* returns all the document type constants that are known in core implementation
* @param prefix a pattern for recognizing constants
* @return an array of type labels
*/
@@ -63,7 +110,7 @@ function search_get_document_types($prefix = 'SEARCH_TYPE_') {
}
sort($ret);
return $ret;
} //search_get_document_types
}
/**
* additional virtual modules to index
@@ -75,10 +122,16 @@ function search_get_document_types($prefix = 'SEARCH_TYPE_') {
*/
function search_get_additional_modules() {
$extras = array(/* additional keywords go here */);
if (defined('SEARCH_EXTRAS')){
$extras = explode(',', SEARCH_EXTRAS);
}
$ret = array();
$temp = new StdClass;
foreach($extras as $extra) {
$temp->name = $extra;
$ret[] = clone($temp);
$temp->location = 'internal';
$ret[$temp->name] = clone($temp);
}
return $ret;
} //search_get_additional_modules
@@ -113,23 +166,6 @@ function search_escape_string($str) {
return $s;
} //search_escape_string
/**
* get a real php 5 version number, using 5.0.0 arbitrarily
* @param feedback if true, prints a feedback message to output.
* @return true if version of PHP is high enough
*/
function search_check_php5($feedback = false) {
if (!check_php_version("5.0.0")) {
if ($feedback) {
print_heading(get_string('versiontoolow', 'search'));
}
return false;
}
else {
return true;
}
} //search_check_php5
/**
* simple timer function, on first call, records a current microtime stamp, outputs result on 2nd call
* @param cli an output formatting switch
search/query.php
+141 -107
View File
@@ -36,6 +36,7 @@
*/
require_once('../config.php');
require_once("$CFG->dirroot/search/lib.php");
// include "debugging.php";
if ($CFG->forcelogin) {
@@ -50,92 +51,105 @@
/// check for php5, but don't die yet (see line 52)
if ($check = search_check_php5()) {
require_once("{$CFG->dirroot}/search/querylib.php");
require_once("{$CFG->dirroot}/search/querylib.php");
$page_number = optional_param('page', -1, PARAM_INT);
$pages = ($page_number == -1) ? false : true;
$advanced = (optional_param('a', '0', PARAM_INT) == '1') ? true : false;
$query_string = stripslashes(optional_param('query_string', '', PARAM_CLEAN));
/// discard harmfull searches
if (!isset($CFG->block_search_utf8dir)){
set_config('block_search_utf8dir', 1);
}
/// discard harmfull searches
if (preg_match("/^[\*\?]+$/", $query_string)){
$query_string = '';
$error = get_string('fullwildcardquery','search');
}
$page_number = optional_param('page', -1, PARAM_INT);
$pages = ($page_number == -1) ? false : true;
$advanced = (optional_param('a', '0', PARAM_INT) == '1') ? true : false;
$query_string = optional_param('query_string', '', PARAM_CLEAN);
if ($pages && isset($_SESSION['search_advanced_query'])) {
// if both are set, then we are busy browsing through the result pages of an advanced query
$adv = unserialize($_SESSION['search_advanced_query']);
}
else if ($advanced) {
// otherwise we are dealing with a new advanced query
unset($_SESSION['search_advanced_query']);
session_unregister('search_advanced_query');
// chars to strip from strings (whitespace)
$chars = " \t\n\r\0\x0B,-+";
// retrieve advanced query variables
$adv->mustappear = trim(optional_param('mustappear', '', PARAM_CLEAN), $chars);
$adv->notappear = trim(optional_param('notappear', '', PARAM_CLEAN), $chars);
$adv->canappear = trim(optional_param('canappear', '', PARAM_CLEAN), $chars);
$adv->module = optional_param('module', '', PARAM_CLEAN);
$adv->title = trim(optional_param('title', '', PARAM_CLEAN), $chars);
$adv->author = trim(optional_param('author', '', PARAM_CLEAN), $chars);
}
if ($advanced) {
//parse the advanced variables into a query string
//TODO: move out to external query class (QueryParse?)
$query_string = '';
// get all available module types
$module_types = array_merge(array('all'), array_values(search_get_document_types()));
$adv->module = in_array($adv->module, $module_types) ? $adv->module : 'all';
// convert '1 2' into '+1 +2' for required words field
if (strlen(trim($adv->mustappear)) > 0) {
$query_string = ' +'.implode(' +', preg_split("/[\s,;]+/", $adv->mustappear));
}
// convert '1 2' into '-1 -2' for not wanted words field
if (strlen(trim($adv->notappear)) > 0) {
$query_string .= ' -'.implode(' -', preg_split("/[\s,;]+/", $adv->notappear));
}
// this field is left untouched, apart from whitespace being stripped
if (strlen(trim($adv->canappear)) > 0) {
$query_string .= ' '.implode(' ', preg_split("/[\s,;]+/", $adv->canappear));
}
// add module restriction
$doctypestr = get_string('doctype', 'search');
$titlestr = get_string('title', 'search');
$authorstr = get_string('author', 'search');
if ($adv->module != 'all') {
$query_string .= " +{$doctypestr}:".$adv->module;
}
// create title search string
if (strlen(trim($adv->title)) > 0) {
$query_string .= " +{$titlestr}:".implode(" +{$titlestr}:", preg_split("/[\s,;]+/", $adv->title));
}
// create author search string
if (strlen(trim($adv->author)) > 0) {
$query_string .= " +{$authorstr}:".implode(" +{$authorstr}:", preg_split("/[\s,;]+/", $adv->author));
}
// save our options if the query is valid
if (!empty($query_string)) {
$_SESSION['search_advanced_query'] = serialize($adv);
}
}
// normalise page number
if ($page_number < 1) {
$page_number = 1;
}
//run the query against the index
$sq = new SearchQuery($query_string, $page_number, 10, false);
if ($pages && isset($_SESSION['search_advanced_query'])) {
// if both are set, then we are busy browsing through the result pages of an advanced query
$adv = unserialize($_SESSION['search_advanced_query']);
} elseif ($advanced) {
// otherwise we are dealing with a new advanced query
unset($_SESSION['search_advanced_query']);
session_unregister('search_advanced_query');
// chars to strip from strings (whitespace)
$chars = " \t\n\r\0\x0B,-+";
// retrieve advanced query variables
$adv->mustappear = trim(optional_param('mustappear', '', PARAM_CLEAN), $chars);
$adv->notappear = trim(optional_param('notappear', '', PARAM_CLEAN), $chars);
$adv->canappear = trim(optional_param('canappear', '', PARAM_CLEAN), $chars);
$adv->module = optional_param('module', '', PARAM_CLEAN);
$adv->title = trim(optional_param('title', '', PARAM_CLEAN), $chars);
$adv->author = trim(optional_param('author', '', PARAM_CLEAN), $chars);
}
if ($advanced) {
//parse the advanced variables into a query string
//TODO: move out to external query class (QueryParse?)
$query_string = '';
// get all available module types adding third party modules
$module_types = array_merge(array('all'), array_values(search_get_document_types()));
$module_types = array_merge($module_types, array_values(search_get_document_types('X_SEARCH_TYPE')));
$adv->module = in_array($adv->module, $module_types) ? $adv->module : 'all';
// convert '1 2' into '+1 +2' for required words field
if (strlen(trim($adv->mustappear)) > 0) {
$query_string = ' +'.implode(' +', preg_split("/[\s,;]+/", $adv->mustappear));
}
// convert '1 2' into '-1 -2' for not wanted words field
if (strlen(trim($adv->notappear)) > 0) {
$query_string .= ' -'.implode(' -', preg_split("/[\s,;]+/", $adv->notappear));
}
// this field is left untouched, apart from whitespace being stripped
if (strlen(trim($adv->canappear)) > 0) {
$query_string .= ' '.implode(' ', preg_split("/[\s,;]+/", $adv->canappear));
}
// add module restriction
$doctypestr = get_string('doctype', 'search');
$titlestr = get_string('title', 'search');
$authorstr = get_string('author', 'search');
if ($adv->module != 'all') {
$query_string .= " +{$doctypestr}:".$adv->module;
}
// create title search string
if (strlen(trim($adv->title)) > 0) {
$query_string .= " +{$titlestr}:".implode(" +{$titlestr}:", preg_split("/[\s,;]+/", $adv->title));
}
// create author search string
if (strlen(trim($adv->author)) > 0) {
$query_string .= " +{$authorstr}:".implode(" +{$authorstr}:", preg_split("/[\s,;]+/", $adv->author));
}
// save our options if the query is valid
if (!empty($query_string)) {
$_SESSION['search_advanced_query'] = serialize($adv);
}
}
// normalise page number
if ($page_number < 1) {
$page_number = 1;
}
//run the query against the index ensuring internal coding works in UTF-8
Zend_Search_Lucene_Analysis_Analyzer::setDefault(new Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8_CaseInsensitive());
$sq = new SearchQuery($query_string, $page_number, 10, false);
if (!$site = get_site()) {
redirect("index.php");
@@ -156,12 +170,9 @@
print_header("$strsearch", "$site->fullname" , $navigation, "", "", true, "&nbsp;", navmenu($site));
}
//keep things pretty, even if php5 isn't available
if (!$check) {
print_heading(search_check_php5(true));
print_footer();
exit(0);
}
if (!empty($error)){
notice ($error);
}
print_box_start();
print_heading($strquery);
@@ -178,12 +189,11 @@
}
}
?>
<form id="query" method="get" action="query.php">
<?php
if (!$advanced) {
?>
<input type="text" name="query_string" length="50" value="<?php print stripslashes($query_string) ?>" />
<input type="text" name="query_string" length="50" value="<?php p($query_string) ?>" />
&nbsp;<input type="submit" value="<?php print_string('search', 'search') ?>" /> &nbsp;
<a href="query.php?a=1"><?php print_string('advancedsearch', 'search') ?></a> |
<a href="stats.php"><?php print_string('statistics', 'search') ?></a>
@@ -271,7 +281,7 @@
</form>
<br/>
<div class="mdl-align">
<div align="center">
<?php
print_string('searching', 'search') . ': ';
@@ -287,7 +297,7 @@
print_string('documents', 'search');
print '.';
if (!$sq->is_valid_index() and isadmin()) {
if (!$sq->is_valid_index() and has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))) {
print '<p>' . get_string('noindexmessage', 'search') . '<a href="indexersplash.php">' . get_string('createanindex', 'search')."</a></p>\n";
}
@@ -296,7 +306,8 @@
<?php
print_box_end();
// prints all the results in a box
/// prints all the results in a box
if ($sq->is_valid()) {
print_box_start();
@@ -305,7 +316,7 @@
print "<br />";
print $hit_count.' '.get_string('resultsreturnedfor', 'search') . " '".stripslashes($query_string)."'.";
print $hit_count.' '.get_string('resultsreturnedfor', 'search') . " '".s($query_string)."'.";
print "<br />";
if ($hit_count > 0) {
@@ -324,30 +335,53 @@
$typestr = get_string('type', 'search');
$scorestr = get_string('score', 'search');
$authorstr = get_string('author', 'search');
foreach ($hits as $listing) {
$searchables = search_collect_searchables(false, false);
foreach ($hits as $listing) {
if ($listing->doctype == 'user'){ // A special handle for users
$icon = print_user_picture ($listing->author, 0, true, 0, true, false) ;
} else {
$iconpath = $CFG->modpixpath.'/'.$listing->doctype.'/icon.gif';
$icon = "<img align=\"top\" src=\"".$iconpath."\" class=\"activityicon\" alt=\"\"/>";
}
$coursename = get_field('course', 'fullname', 'id', $listing->courseid);
$courseword = mb_convert_case(get_string('course', 'moodle'), MB_CASE_LOWER, 'UTF-8');
$course = ($listing->doctype != 'user') ? '<strong> ('.$courseword.': \''.$coursename.'\')</strong>' : '' ;
//if ($CFG->unicodedb) {
//$listing->title = mb_convert_encoding($listing->title, 'auto', 'UTF8');
//}
$title_post_processing_function = $listing->doctype.'_link_post_processing';
require_once "{$CFG->dirroot}/search/documents/{$listing->doctype}_document.php";
$searchable_instance = $searchables[$listing->doctype];
if ($searchable_instance->location == 'internal'){
require_once "{$CFG->dirroot}/search/documents/{$listing->doctype}_document.php";
} else {
require_once "{$CFG->dirroot}/{$searchable_instance->location}/{$listing->doctype}/search_document.php";
}
if (function_exists($title_post_processing_function)) {
$listing->title = $title_post_processing_function($listing->title);
}
print "<li value='".($listing->number+1)."'><a href='".str_replace('DEFAULT_POPUP_SETTINGS', DEFAULT_POPUP_SETTINGS ,$listing->url)."'>$listing->title</a><br />\n"
."<em>".search_shorten_url($listing->url, 70)."</em><br />\n"
."{$typestr}: ".$listing->doctype.", {$scorestr}: ".round($listing->score, 3);
if (!empty($listing->author)){
print ", {$authorstr}: ".$listing->author."\n"
echo "<li value='".($listing->number + 1)."'><a href='"
.str_replace('DEFAULT_POPUP_SETTINGS', DEFAULT_POPUP_SETTINGS ,$listing->url)
."'>$icon $listing->title</a> $course<br />\n";
// print "<li value='".($listing->number+1)."'><a href='".str_replace('DEFAULT_POPUP_SETTINGS', DEFAULT_POPUP_SETTINGS ,$listing->url)."'>$listing->title</a><br />\n"
// ."<em>".search_shorten_url($listing->url, 70)."</em><br />\n"
echo "{$typestr}: " . $listing->doctype . ", {$scorestr}: " . round($listing->score, 3);
if (!empty($listing->author) && !is_numeric($listing->author)){
echo ", {$authorstr}: ".$listing->author."\n"
."</li>\n";
}
}
print "</ol>";
print $page_links;
echo "</ol>";
echo $page_links;
}
print_box_end();
?>
<div style="text-align:center">
<div align="center">
<?php
print_string('ittook', 'search');
search_stopwatch();
@@ -359,4 +393,4 @@
}
print_box_end();
print_footer();
?>
?>
search/querylib.php
+93 -74
View File
@@ -26,7 +26,8 @@ public $url,
$doctype,
$author,
$score,
$number;
$number,
$courseid;
}
@@ -67,17 +68,17 @@ private $mode,
$last_term = $this->fetch('search_last_term');
//if this query is different from the last, clear out the last one
if ($id != false and $last_term != $id) {
if ($id != false && $last_term != $id) {
$this->clear($last_term);
}
//store the new query if id and object are passed in
if ($object and $id) {
if ($object && $id) {
$this->store('search_last_term', $id);
$this->store($id, $object);
return true;
//otherwise return the stored results
} else if ($id and $this->exists($id)) {
} else if ($id && $this->exists($id)) {
return $this->fetch($id);
}
}
@@ -219,13 +220,17 @@ class SearchQuery {
private function process_results($all=false) {
global $USER;
$term = mb_convert_case($this->term, MB_CASE_LOWER, 'UTF-8');
// $term = mb_convert_case($this->term, MB_CASE_LOWER, 'UTF-8');
$term = $this->term;
$page = optional_param('page', 1, PARAM_INT);
//experimental - return more results
$strip_arr = array('author:', 'title:', '+', '-', 'doctype:');
$stripped_term = str_replace($strip_arr, '', $term);
// $strip_arr = array('author:', 'title:', '+', '-', 'doctype:');
// $stripped_term = str_replace($strip_arr, '', $term);
$hits = $this->index->find($term." title:".$stripped_term." author:".$stripped_term);
// $search_string = $term." title:".$stripped_term." author:".$stripped_term;
$search_string = $term;
$hits = $this->index->find($search_string);
//--
$hitcount = count($hits);
@@ -233,49 +238,58 @@ class SearchQuery {
if ($hitcount == 0) return array();
$totalpages = ceil($hitcount/$this->results_per_page);
$resultdoc = new SearchResult();
$resultdocs = array();
$searchables = search_collect_searchables(false, false);
$realindex = 0;
/**
if (!$all) {
if ($hitcount < $this->results_per_page) {
if ($finalresults < $this->results_per_page) {
$this->pagenumber = 1;
} else if ($this->pagenumber > $totalpages) {
} elseif ($this->pagenumber > $totalpages) {
$this->pagenumber = $totalpages;
}
$start = ($this->pagenumber - 1) * $this->results_per_page;
$end = $start + $this->results_per_page;
if ($end > $hitcount) {
$end = $hitcount;
if ($end > $finalresults) {
$end = $finalresults;
}
} else {
$start = 0;
$end = $hitcount;
}
$end = $finalresults;
} */
$resultdoc = new SearchResult();
$resultdocs = array();
for ($i = $start; $i < $end; $i++) {
for ($i = 0; $i < min($hitcount, ($page) * $this->results_per_page); $i++) {
$hit = $hits[$i];
//check permissions on each result
if ($this->can_display($USER, $hit->docid, $hit->doctype, $hit->course_id, $hit->group_id, $hit->path, $hit->itemtype, $hit->context_id )) {
$resultdoc->number = $i;
$resultdoc->url = $hit->url;
$resultdoc->title = $hit->title;
$resultdoc->score = $hit->score;
$resultdoc->doctype = $hit->doctype;
$resultdoc->author = $hit->author;
//and store it
$resultdocs[] = clone($resultdoc);
if ($this->can_display($USER, $hit->docid, $hit->doctype, $hit->course_id, $hit->group_id, $hit->path, $hit->itemtype, $hit->context_id, $searchables )) {
if ($i >= ($page - 1) * $this->results_per_page){
$resultdoc->number = $realindex;
$resultdoc->url = $hit->url;
$resultdoc->title = $hit->title;
$resultdoc->score = $hit->score;
$resultdoc->doctype = $hit->doctype;
$resultdoc->author = $hit->author;
$resultdoc->courseid = $hit->course_id;
//and store it
$resultdocs[] = clone($resultdoc);
}
$realindex++;
} else {
// lowers total_results one unit
$this->total_results--;
}
}
$totalpages = ceil($this->total_results/$this->results_per_page);
return $resultdocs;
}
@@ -286,7 +300,7 @@ class SearchQuery {
private function get_results() {
$cache = new SearchCache();
if ($this->cache and $cache->can_cache()) {
if ($this->cache && $cache->can_cache()) {
if (!($resultdocs = $cache->cache($this->term))) {
$resultdocs = $this->process_results();
//cache the results so we don't have to compute this on every page-load
@@ -298,7 +312,7 @@ class SearchQuery {
}
} else {
//no caching :(
//print "Caching disabled!";
// print "Caching disabled!";
$resultdocs = $this->process_results();
}
return $resultdocs;
@@ -315,7 +329,7 @@ class SearchQuery {
$next = get_string('next', 'search');
$back = get_string('back', 'search');
$ret = "<div class='mdl-align' id='search_page_links'>";
$ret = "<div align='center' id='search_page_links'>";
//Back is disabled if we're on page 1
if ($page > 1) {
@@ -370,56 +384,61 @@ class SearchQuery {
* @uses CFG
* // TODO reorder parameters more consistently
*/
private function can_display(&$user, $this_id, $doctype, $course_id, $group_id, $path, $item_type, $context_id) {
private function can_display(&$user, $this_id, $doctype, $course_id, $group_id, $path, $item_type, $context_id, &$searchables) {
global $CFG;
/**
* course related checks
*/
// admins can see everything, anyway.
if (isadmin()){
if (has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))){
return true;
}
// first check course compatibility against user : enrolled users to that course can see.
$myCourses = get_my_courses($user->id);
$unenroled = !in_array($course_id, array_keys($myCourses));
// if guests are allowed, logged guest can see
$isallowedguest = (isguest()) ? get_field('course', 'guest', 'id', $course_id) : false ;
if ($unenroled && !$isallowedguest){
return false;
}
// if user is enrolled or is allowed user and course is hidden, can he see it ?
$visibility = get_field('course', 'visible', 'id', $course_id);
if ($visibility <= 0){
if (!has_capability('moodle/course:viewhiddencourses', get_context_instance(CONTEXT_COURSE, $course_id))){
return false;
}
}
/**
* prerecorded capabilities
*/
// get context caching information and tries to discard unwanted records here
/**
* final checks
*/
// then give back indexing data to the module for local check
include_once "{$CFG->dirroot}/search/documents/{$doctype}_document.php";
$access_check_function = "{$doctype}_check_text_access";
if (function_exists($access_check_function)){
$modulecheck = $access_check_function($path, $item_type, $this_id, $user, $group_id, $context_id);
// echo "module said $modulecheck for item $doctype/$item_type/$this_id";
return($modulecheck);
}
// first check course compatibility against user : enrolled users to that course can see.
$myCourses = get_my_courses($user->id);
$unenroled = !in_array($course_id, array_keys($myCourses));
return true;
// if guests are allowed, logged guest can see
$isallowedguest = (isguest()) ? get_field('course', 'guest', 'id', $course_id) : false ;
if ($unenroled && !$isallowedguest){
return false;
}
// if user is enrolled or is allowed user and course is hidden, can he see it ?
$visibility = get_field('course', 'visible', 'id', $course_id);
if ($visibility <= 0){
if (!has_capability('moodle/course:viewhiddencourses', get_context_instance(CONTEXT_COURSE, $course_id))){
return false;
}
}
/**
* prerecorded capabilities
*/
// get context caching information and tries to discard unwanted records here
/**
* final checks
*/
// then give back indexing data to the module for local check
$searchable_instance = $searchables[$doctype];
if ($searchable_instance->location == 'internal'){
include_once "{$CFG->dirroot}/search/documents/{$doctype}_document.php";
} else {
include_once "{$CFG->dirroot}/{$searchable_instance->location}/$doctype/search_document.php";
}
$access_check_function = "{$doctype}_check_text_access";
if (function_exists($access_check_function)){
$modulecheck = $access_check_function($path, $item_type, $this_id, $user, $group_id, $context_id);
// echo "module said $modulecheck for item $doctype/$item_type/$this_id";
return($modulecheck);
}
return true;
}
/**
@@ -471,4 +490,4 @@ class SearchQuery {
return $this->results_per_page;
}
}
?>
?>
search/searchtypes.php
+34
View File
@@ -0,0 +1,34 @@
<?php
/**
* Global Search Engine for Moodle
*
* @package search
* @category core
* @subpackage search_engine
* @author Michael Champanis (mchampan) [[email protected]], Valery Fremaux [[email protected]] > 1.8
* @date 2008/03/31
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*
* Searcheable types
* to disable a type, just comment the two declaration lines for that type
*
*/
//document types that can be searched
//define('SEARCH_TYPE_NONE', 'none');
define('SEARCH_TYPE_WIKI', 'wiki');
define('SEARCH_TYPE_FORUM', 'forum');
define('SEARCH_TYPE_GLOSSARY', 'glossary');
define('SEARCH_TYPE_RESOURCE', 'resource');
define('SEARCH_TYPE_DATA', 'data');
define('SEARCH_TYPE_CHAT', 'chat');
define('SEARCH_TYPE_LESSON', 'lesson');
define('SEARCH_TYPE_ASSIGNMENT', 'assignment');
define('SEARCH_TYPE_LABEL', 'label');
define('SEARCH_EXTRAS', 'user');
define('SEARCH_TYPE_USER', 'user');
define('PATH_FOR_SEARCH_TYPE_USER', 'user');
?>
search/stats.php
+4 -12
View File
@@ -32,11 +32,9 @@ require_once("{$CFG->dirroot}/search/lib.php");
/// check for php5, but don't die yet
if ($check = search_check_php5()) {
require_once("{$CFG->dirroot}/search/indexlib.php");
require_once("{$CFG->dirroot}/search/indexlib.php");
$indexinfo = new IndexInfo();
}
$indexinfo = new IndexInfo();
if (!$site = get_site()) {
redirect("index.php");
@@ -58,12 +56,6 @@ require_once("{$CFG->dirroot}/search/lib.php");
/// keep things pretty, even if php5 isn't available
if (!$check) {
print_heading(search_check_php5(true));
print_footer();
exit(0);
}
print_box_start();
print_heading($strquery);
@@ -77,7 +69,7 @@ require_once("{$CFG->dirroot}/search/lib.php");
/// this table is only for admins, shows index directory size and location
if (isadmin()) {
if (has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))) {
$datadirectorystr = get_string('datadirectory', 'search');
$inindexdirectorystr = get_string('filesinindexdirectory', 'search');
$totalsizestr = get_string('totalsize', 'search');
@@ -147,7 +139,7 @@ require_once("{$CFG->dirroot}/search/lib.php");
/// add extra fields if we're admin
if (isadmin()) {
if (has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))) {
//don't want to confuse users if the two totals don't match (hint: they should)
$table->data[] = array($documentsinindexstr, $indexinfo->indexcount);
search/update.php
+37 -36
View File
@@ -1,25 +1,31 @@
<?php
/**
* Global Search Engine for Moodle
*
* @package search
* @category core
* @subpackage search_engine
* @author Michael Champanis (mchampan) [[email protected]], Valery Fremaux [[email protected]] > 1.8
* @date 2008/03/31
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*
* Index asynchronous updator
*
* Major chages in this review is passing the xxxx_db_names return to
* multiple arity to handle multiple document types modules
*/
/**
* Global Search Engine for Moodle
*
* @package search
* @category core
* @subpackage search_engine
* @author Michael Champanis (mchampan) [[email protected]], Valery Fremaux [[email protected]] > 1.8
* @date 2008/03/31
* @license http://www.gnu.org/copyleft/gpl.html GNU Public License
*
* Index asynchronous updator
*
* Major chages in this review is passing the xxxx_db_names return to
* multiple arity to handle multiple document types modules
*/
/**
* includes and requires
*/
require_once('../config.php');
/**
* includes and requires
*/
require_once('../config.php');
require_once("$CFG->dirroot/search/lib.php");
/// makes inclusions of the Zend Engine more reliable
$separator = (array_key_exists('WINDIR', $_SERVER)) ? ';' : ':' ;
ini_set('include_path', $CFG->dirroot.'/search'.$separator.ini_get('include_path'));require_login();
require_once("$CFG->dirroot/search/lib.php");
require_once("$CFG->dirroot/search/indexlib.php");
/// checks global search activation
@@ -29,24 +35,19 @@ require_once("$CFG->dirroot/search/lib.php");
error(get_string('globalsearchdisabled', 'search'));
}
if (!isadmin()) {
if (!has_capability('moodle/site:doanything', get_context_instance(CONTEXT_SYSTEM))) {
error(get_string('beadmin', 'search'), "$CFG->wwwroot/login/index.php");
}
/// check for php5 (lib.php)
if (!search_check_php5()) {
$phpversion = phpversion();
mtrace("Sorry, global search requires PHP 5.0.0 or later (currently using version ".phpversion().")");
exit(0);
}
require_once("$CFG->dirroot/search/indexlib.php");
$index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
try {
$index = new Zend_Search_Lucene(SEARCH_INDEX_PATH);
} catch(LuceneException $e) {
mtrace("Could not construct a valid index. Maybe the first indexation was never made, or files might be corrupted. Run complete indexation again.");
return;
}
$dbcontrol = new IndexDBControl();
$update_count = 0;
$indexdate = $CFG->search_indexer_update_date;
$indexdate = @$CFG->search_indexer_update_date;
$startupdatedate = time();
/// indexing changed resources
@@ -73,7 +74,7 @@ require_once("$CFG->dirroot/search/lib.php");
if ($valuesArray){
foreach($valuesArray as $values){
$where = (isset($values[5])) ? 'AND ('.$values[5].')' : '';
$where = (!empty($values[5])) ? 'AND ('.$values[5].')' : '';
$itemtypes = ($values[4] != '*' && $values[4] != 'any') ? " AND itemtype = '{$values[4]}' " : '' ;
//TODO: check 'in' syntax with other RDBMS' (add and update.php as well)
@@ -85,10 +86,10 @@ require_once("$CFG->dirroot/search/lib.php");
FROM
{$CFG->prefix}{$table}
WHERE
doctype = '{$mod->name}'
doctype = '{$values[1]}'
$itemtypes
";
$docIds = get_records_sql_menu($query);
$docIds = get_records_sql_menu($query, array($mod->name));
$docIdList = ($docIds) ? implode("','", array_keys($docIds)) : '' ;
$query = "