Skip to content

Commit 6f1845b

Browse files
author
[email protected]
committed
[Packaging] Adding Solr support and pyenvs for 1.5b and CKAN workshop
1 parent 4575db6 commit 6f1845b

File tree

8 files changed

+158
-12
lines changed

8 files changed

+158
-12
lines changed

‎LICENSE.txt

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
License
2+
+++++++
3+
4+
CKAN - Data Catalogue Software
5+
Copyright (C) 2007 Open Knowledge Foundation
6+
7+
This program is free software: you can redistribute it and/or modify
8+
it under the terms of the GNU Affero General Public License as
9+
published by the Free Software Foundation, either version 3 of the
10+
License, or (at your option) any later version.
11+
12+
This program is distributed in the hope that it will be useful,
13+
but WITHOUT ANY WARRANTY; without even the implied warranty of
14+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
15+
GNU Affero General Public License for more details.
16+
17+
You should have received a copy of the GNU Affero General Public License
18+
along with this program. If not, see <http://www.gnu.org/licenses/>.
19+
20+
21+
Note
22+
====
23+
24+
CKAN is sometimes packaged directly with other software (listed in
25+
requires/lucid_conflict.txt). In these cases, we are required to list the
26+
licenses of the packaged softare too. They are all AGPL compatible and read as
27+
follows in the next sections.
28+
29+
WebHelpers
30+
----------
31+
32+
Portions of WebHelpers covered by the following license::
33+
34+
Copyright (c) 2005, the Lawrence Journal-World
35+
All rights reserved.
36+
37+
Redistribution and use in source and binary forms, with or without modification,
38+
are permitted provided that the following conditions are met:
39+
40+
1. Redistributions of source code must retain the above copyright notice,
41+
this list of conditions and the following disclaimer.
42+
43+
2. Redistributions in binary form must reproduce the above copyright
44+
notice, this list of conditions and the following disclaimer in the
45+
documentation and/or other materials provided with the distribution.
46+
47+
3. Neither the name of Django nor the names of its contributors may be used
48+
to endorse or promote products derived from this software without
49+
specific prior written permission.
50+
51+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
52+
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
53+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
54+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
55+
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
56+
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
57+
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
58+
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
59+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
60+
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
61+
62+
Genshi
63+
------
64+
65+
Copyright © 2006-2007 Edgewall Software
66+
All rights reserved.
67+
68+
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
69+
70+
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
71+
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
72+
3. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission.
73+
74+
THIS SOFTWARE IS PROVIDED BY THE AUTHOR “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
75+
76+
Migrate
77+
-------
78+
79+
Open Source Initiative OSI - The MIT License:Licensing
80+
[OSI Approved License]
81+
82+
The MIT License
83+
84+
Copyright (c) <year> <copyright holders>
85+
86+
Permission is hereby granted, free of charge, to any person obtaining a copy
87+
of this software and associated documentation files (the "Software"), to deal
88+
in the Software without restriction, including without limitation the rights
89+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
90+
copies of the Software, and to permit persons to whom the Software is
91+
furnished to do so, subject to the following conditions:
92+
93+
The above copyright notice and this permission notice shall be included in
94+
all copies or substantial portions of the Software.
95+
96+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
97+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
98+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
99+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
100+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
101+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
102+
THE SOFTWARE.
103+
104+
SQLAlchemy
105+
----------
106+
107+
This is the MIT license: http://www.opensource.org/licenses/mit-license.php
108+
109+
Copyright (c) 2005-2011 Michael Bayer and contributors. SQLAlchemy is a trademark of Michael Bayer.
110+
111+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
112+
113+
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
114+
115+
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
116+
117+

‎MANIFEST.in

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,9 @@
11
include ckan/config/deployment.ini_tmpl
22
recursive-include ckan/public *
33
recursive-include ckan/config *.ini
4+
recursive-include ckan/config *.xml
45
recursive-include ckan/templates *
56
recursive-include ckan *.ini
67
prune .hg
78
include CHANGELOG.txt
8-
include ckan/migration/migrate.cfg
9+
include ckan/migration/migrate.cfg

‎ckan/config/deployment.ini_tmpl

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,8 @@ ckan.site_url =
143143
## Favicon (default is the CKAN software favicon)
144144
ckan.favicon = http://assets.okfn.org/p/ckan/img/ckan.ico
145145

146+
## Solr support
147+
#solr_url = http://127.0.0.1:8983/solr
146148

147149
## An 'id' for the site (using, for example, when creating entries in a common search index)
148150
## If not specified derived from the site_url

‎ckan_deb/DEBIAN/control.template

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Architecture: all
55
Maintainer: James Gardner <[email protected]>
66
Section: main/web
77
Priority: extra
8-
Depends: python-ckan, postgresql-8.4, apache2, libapache2-mod-wsgi, python-apachemiddleware, python-virtualenv, python-pip
8+
Depends: python-ckan, postgresql-8.4, apache2, libapache2-mod-wsgi, python-virtualenv, python-pip, solr-jetty
99
Homepage: http://ckan.org
1010
Description: ckan
1111
Common files useful for managing CKAN installations

‎ckan_deb/usr/bin/ckan-create-instance

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,8 @@ echo "Making sure PostgreSQL is running ..."
4343
echo "Ensuring users and groups are set up correctly ..."
4444
ckan_ensure_users_and_groups ${INSTANCE}
4545

46+
chown ckan${INSTANCE}:ckan${INSTANCE} /etc/ckan/${INSTANCE}/install_settings.sh
47+
4648
echo "Ensuring directories exist for ${INSTANCE} CKAN INSTANCE ..."
4749
ckan_make_ckan_directories ${INSTANCE}
4850

@@ -75,6 +77,17 @@ ckan_create_wsgi_handler ${INSTANCE}
7577
echo "Setting the password of the ${INSTANCE} user in PostgreSQL"
7678
ckan_add_or_replace_database_user ${INSTANCE} ${CKAN_DB_PASSWORD}
7779

80+
# Solr support
81+
echo "Setting up Solr ..."
82+
sed \
83+
-e "s,NO_START=1,NO_START=0," \
84+
-e "s,#JETTY_HOST=\$(uname -n),JETTY_HOST=127.0.0.1," \
85+
-e "s,#JETTY_PORT=8080,JETTY_PORT=8983," \
86+
-i /etc/default/jetty
87+
mv /usr/share/solr/conf/schema.xml /usr/share/solr/conf/schema.xml.`date --utc "+%Y-%m-%d_%T"`.bak
88+
ln -s /usr/lib/pymodules/python2.6/ckan/config/schema.xml /usr/share/solr/conf/schema.xml
89+
service jetty start
90+
7891
# Create the config file
7992
echo "Creating/overwriting the config for CKAN ... "
8093
# We use the actual password in PostgreSQL in case any non-sense has gone on
@@ -121,7 +134,7 @@ echo "Restarting apache ..."
121134
# Install the new crontab
122135
echo "Enabling crontab for the ckan${INSTANCE} user ..."
123136
PACKAGED_CRONJOB="/tmp/${INSTANCE}-cronjob"
124-
cat <<EOF > ${PACKAGED_CRONJOB}
137+
cat << EOF > ${PACKAGED_CRONJOB}
125138
# WARNING: Do not edit these cron tabs, they will be overwritten any time
126139
# the ckan INSTANCE package is upgraded
127140
# QUESTION: Should email reports be sent to root?

‎ckan_deb/usr/lib/ckan/common.sh

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,12 +45,12 @@ ckan_ensure_users_and_groups () {
4545
INSTANCE=$1
4646
COMMAND_OUTPUT=`cat /etc/group | grep "ckan${INSTANCE}:"`
4747
if ! [[ "$COMMAND_OUTPUT" =~ "ckan${INSTANCE}:" ]] ; then
48-
echo "Crating the 'ckan${INSTANCE}' group ..."
48+
echo "Creating the 'ckan${INSTANCE}' group ..."
4949
sudo groupadd --system "ckan${INSTANCE}"
5050
fi
5151
COMMAND_OUTPUT=`cat /etc/passwd | grep "ckan${INSTANCE}:"`
5252
if ! [[ "$COMMAND_OUTPUT" =~ "ckan${INSTANCE}:" ]] ; then
53-
echo "Crating the 'ckan${INSTANCE}' user ..."
53+
echo "Creating the 'ckan${INSTANCE}' user ..."
5454
sudo useradd --system --gid "ckan${INSTANCE}" --home /var/lib/ckan/${INSTANCE} -M --shell /usr/sbin/nologin ckan${INSTANCE}
5555
fi
5656
}
@@ -84,6 +84,7 @@ ckan_create_who_ini () {
8484
cp -n /usr/share/pyshared/ckan/config/who.ini /etc/ckan/${INSTANCE}/who.ini
8585
sed -e "s,%(here)s,/var/lib/ckan/${INSTANCE}," \
8686
-i /etc/ckan/${INSTANCE}/who.ini
87+
chown ckan${INSTANCE}:ckan${INSTANCE} /etc/ckan/${INSTANCE}/who.ini
8788
fi
8889
fi
8990
}
@@ -109,7 +110,11 @@ ckan_create_config_file () {
109110
-e "s,^\(sqlalchemy.url\)[ =].*,\1 = postgresql://${INSTANCE}:${password}@localhost/${INSTANCE}," \
110111
-e "s,ckan\.site_logo,\#ckan.site_logo," \
111112
-e "s,ckan\.log,/var/log/ckan/${INSTANCE}/${INSTANCE}.log," \
113+
-e "s,ckan\.site_id,${INSTANCE}," \
114+
-e "s,ckan\.site_description,${INSTANCE}," \
115+
-e "s,#solr_url = http://127.0.0.1:8983/solr,solr_url = http://127.0.0.1:8983/solr," \
112116
-i /etc/ckan/${INSTANCE}/${INSTANCE}.ini
117+
sudo chown ckan${INSTANCE}:ckan${INSTANCE} /etc/ckan/${INSTANCE}/${INSTANCE}.ini
113118
fi
114119
}
115120

@@ -147,6 +152,7 @@ ckan_ensure_db_exists () {
147152
if ! [[ "$COMMAND_OUTPUT" =~ ${INSTANCE} ]] ; then
148153
echo "Creating the database ..."
149154
sudo -u postgres createdb -O ${INSTANCE} ${INSTANCE}
155+
paster --plugin=ckan db init --config=/etc/ckan/${INSTANCE}/${INSTANCE}.ini
150156
fi
151157
fi
152158
}
@@ -161,7 +167,11 @@ ckan_create_wsgi_handler () {
161167
INSTANCE=$1
162168
if [ ! -f "/var/lib/ckan/${INSTANCE}/wsgi.py" ]
163169
then
164-
sudo virtualenv --setuptools /var/lib/ckan/${INSTANCE}/pyenv
170+
sudo mkdir /var/lib/ckan/${INSTANCE}/pyenv
171+
sudo chown -R ckan${INSTANCE}:ckan${INSTANCE} /var/lib/ckan/${INSTANCE}/pyenv
172+
sudo -u ckan${INSTANCE} virtualenv --setuptools /var/lib/ckan/${INSTANCE}/pyenv
173+
echo "Attempting to install Pip 1.0 from pypi.python.org into pyenv to be used for extensions ..."
174+
sudo -u ckan${INSTANCE} /var/lib/ckan/${INSTANCE}/pyenv/bin/easy_install pip -U "pip>=1.0" "pip<=1.0.99"
165175
cat <<- EOF > /var/lib/ckan/${INSTANCE}/wsgi.py
166176
import os
167177
instance_dir = '/var/lib/ckan/${INSTANCE}'
@@ -235,6 +245,10 @@ ckan_overwrite_apache_config () {
235245
# pass authorization info on (needed for rest api)
236246
WSGIPassAuthorization On
237247
248+
# Deploy as a daemon (avoids conflicts between CKAN instances)
249+
# WSGIDaemonProcess ${INSTANCE} display-name=${INSTANCE} processes=4 threads=15 maximum-requests=10000
250+
# WSGIProcessGroup ${INSTANCE}
251+
238252
ErrorLog /var/log/apache2/${INSTANCE}.error.log
239253
CustomLog /var/log/apache2/${INSTANCE}.custom.log combined
240254
EOF

‎requires/lucid_missing.txt

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,17 +8,17 @@
88
# vdm>=0.9,<0.9.99
99
-e hg+https://bitbucket.org/okfn/[email protected]#egg=vdm
1010
# markupsafe==0.9.2 required by webhelpers==1.2 required by formalchemy with SQLAlchemy 0.6
11-
-e git+https://github.com/mitsuhiko/markupsafe.git@0.9.2#egg=markupsafe
11+
markupsafe==0.9.2
1212
# autoneg>=0.5
1313
-e git+https://github.com/wwaites/autoneg.git@b4c727b164f411cc9d60#egg=autoneg
1414
# flup>=0.5
1515
-e hg+http://hg.saddi.com/flup@301a58656bfb#egg=flup
1616
# solrpy == 0.9.4
17-
solrpy==0.9.4
18-
# All the conflicting dependencies from the lucid_conflict.txt file
19-
-e hg+https://bitbucket.org/okfn/ckan-deps@6287665a1965#egg=ckan-deps
17+
#solrpy==0.9.4
2018
# FormAlchemy
21-
-e git+https://github.com/FormAlchemy/[email protected]#egg=formalchemy
19+
formalchemy==1.3.9
20+
# Apachemiddleware
21+
-e hg+https://hg.3aims.com/public/ApacheMiddleware@tip#egg=apachemiddleware
2222

2323
# NOTE: Developers, our build script for the Debian packages relies on the
2424
# requirements above being specified as editable resources with their

‎setup.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@
2020
install_requires=[
2121
],
2222
extras_require = {
23-
'solr': ['solrpy==0.9.4'],
2423
},
2524
packages=find_packages(exclude=['ez_setup']),
2625
include_package_data=True,

0 commit comments

Comments
 (0)