diff --git a/HACKING.rst b/HACKING.rst index 5c15537915..83455e3638 100644 --- a/HACKING.rst +++ b/HACKING.rst @@ -20,7 +20,7 @@ in `How To Contribute`__ in the OpenStack wiki. `DevStack's LaunchPad project`_ contains the usual links for blueprints, bugs, tec. __ contribute_ -.. _contribute: http://wiki.openstack.org/HowToContribute. +.. _contribute: http://wiki.openstack.org/HowToContribute __ lp_ .. _lp: https://launchpad.net/~devstack @@ -231,7 +231,7 @@ These scripts are executed serially by ``exercise.sh`` in testing situations. Bash Style Guidelines ~~~~~~~~~~~~~~~~~~~~~ -Devstack defines a bash set of best practices for maintaining large +DevStack defines a bash set of best practices for maintaining large collections of bash scripts. These should be considered as part of the review process. diff --git a/README.md b/README.md index 89e3855d38..f0406c6c7f 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,16 @@ DevStack is a set of scripts and utilities to quickly deploy an OpenStack cloud. # Goals -* To quickly build dev OpenStack environments in a clean Ubuntu or Fedora environment -* To describe working configurations of OpenStack (which code branches work together? what do config files look like for those branches?) -* To make it easier for developers to dive into OpenStack so that they can productively contribute without having to understand every part of the system at once +* To quickly build dev OpenStack environments in a clean Ubuntu or Fedora + environment +* To describe working configurations of OpenStack (which code branches + work together? what do config files look like for those branches?) +* To make it easier for developers to dive into OpenStack so that they can + productively contribute without having to understand every part of the + system at once * To make it easy to prototype cross-project features -* To provide an environment for the OpenStack CI testing on every commit to the projects +* To provide an environment for the OpenStack CI testing on every commit + to the projects Read more at http://devstack.org. @@ -42,12 +47,14 @@ your VM. To start a dev cloud run the following NOT AS ROOT (see ./stack.sh -When the script finishes executing, you should be able to access OpenStack endpoints, like so: +When the script finishes executing, you should be able to access OpenStack +endpoints, like so: * Horizon: http://myhost/ * Keystone: http://myhost:5000/v2.0/ -We also provide an environment file that you can use to interact with your cloud via CLI: +We also provide an environment file that you can use to interact with your +cloud via CLI: # source openrc file to load your environment with OpenStack CLI creds . openrc @@ -63,7 +70,12 @@ If the EC2 API is your cup-o-tea, you can create credentials and use euca2ools: # DevStack Execution Environment -DevStack runs rampant over the system it runs on, installing things and uninstalling other things. Running this on a system you care about is a recipe for disappointment, or worse. Alas, we're all in the virtualization business here, so run it in a VM. And take advantage of the snapshot capabilities of your hypervisor of choice to reduce testing cycle times. You might even save enough time to write one more feature before the next feature freeze... +DevStack runs rampant over the system it runs on, installing things and +uninstalling other things. Running this on a system you care about is a recipe +for disappointment, or worse. Alas, we're all in the virtualization business +here, so run it in a VM. And take advantage of the snapshot capabilities +of your hypervisor of choice to reduce testing cycle times. You might even save +enough time to write one more feature before the next feature freeze... ``stack.sh`` needs to have root access for a lot of tasks, but uses ``sudo`` for all of those tasks. However, it needs to be not-root for most of its @@ -93,7 +105,8 @@ See the **Local Configuration** section below for more details. # Database Backend -Multiple database backends are available. The available databases are defined in the lib/databases directory. +Multiple database backends are available. The available databases are defined +in the lib/databases directory. `mysql` is the default database, choose a different one by putting the following in the `localrc` section: @@ -124,7 +137,7 @@ Apache web server is enabled for wsgi services by setting `APACHE_ENABLED_SERVICES` in your ``localrc`` section. Remember to enable these services at first as above. - APACHE_ENABLED_SERVICES+=keystone,swift + APACHE_ENABLED_SERVICES+=key,swift # Swift @@ -235,20 +248,24 @@ To change this, set the `Q_AGENT` variable to the agent you want to run (e.g. linuxbridge). Variable Name Notes - ------------------------------------------------------------------------------------- - Q_AGENT This specifies which agent to run with the ML2 Plugin (either `openvswitch` or `linuxbridge`). - Q_ML2_PLUGIN_MECHANISM_DRIVERS The ML2 MechanismDrivers to load. The default is none. Note, ML2 will work with the OVS and LinuxBridge agents by default. - Q_ML2_PLUGIN_TYPE_DRIVERS The ML2 TypeDrivers to load. Defaults to all available TypeDrivers. + ---------------------------------------------------------------------------- + Q_AGENT This specifies which agent to run with the + ML2 Plugin (either `openvswitch` or `linuxbridge`). + Q_ML2_PLUGIN_MECHANISM_DRIVERS The ML2 MechanismDrivers to load. The default + is none. Note, ML2 will work with the OVS + and LinuxBridge agents by default. + Q_ML2_PLUGIN_TYPE_DRIVERS The ML2 TypeDrivers to load. Defaults to + all available TypeDrivers. Q_ML2_PLUGIN_GRE_TYPE_OPTIONS GRE TypeDriver options. Defaults to none. Q_ML2_PLUGIN_VXLAN_TYPE_OPTIONS VXLAN TypeDriver options. Defaults to none. Q_ML2_PLUGIN_VLAN_TYPE_OPTIONS VLAN TypeDriver options. Defaults to none. # Heat -Heat is disabled by default. To enable it you'll need the following settings -in your `localrc` section: +Heat is enabled by default (see `stackrc` file). To disable it explicitly +you'll need the following settings in your `localrc` section: - enable_service heat h-api h-api-cfn h-api-cw h-eng + disable_service heat h-api h-api-cfn h-api-cw h-eng Heat can also run in standalone mode, and be configured to orchestrate on an external OpenStack cloud. To launch only Heat in standalone mode @@ -262,14 +279,16 @@ you'll need the following settings in your `localrc` section: # Tempest -If tempest has been successfully configured, a basic set of smoke tests can be run as follows: +If tempest has been successfully configured, a basic set of smoke +tests can be run as follows: $ cd /opt/stack/tempest $ nosetests tempest/scenario/test_network_basic_ops.py # DevStack on Xenserver -If you would like to use Xenserver as the hypervisor, please refer to the instructions in `./tools/xen/README.md`. +If you would like to use Xenserver as the hypervisor, please refer +to the instructions in `./tools/xen/README.md`. # Additional Projects @@ -283,8 +302,10 @@ for more information. # Multi-Node Setup -A more interesting setup involves running multiple compute nodes, with Neutron networks connecting VMs on different compute nodes. -You should run at least one "controller node", which should have a `stackrc` that includes at least: +A more interesting setup involves running multiple compute nodes, with Neutron +networks connecting VMs on different compute nodes. +You should run at least one "controller node", which should have a `stackrc` +that includes at least: disable_service n-net enable_service q-svc @@ -299,7 +320,8 @@ will balance VMs across hosts: SCHEDULER=nova.scheduler.simple.SimpleScheduler -You can then run many compute nodes, each of which should have a `stackrc` which includes the following, with the IP address of the above controller node: +You can then run many compute nodes, each of which should have a `stackrc` +which includes the following, with the IP address of the above controller node: ENABLED_SERVICES=n-cpu,rabbit,g-api,neutron,q-agt SERVICE_HOST=[IP of controller node] @@ -310,22 +332,31 @@ You can then run many compute nodes, each of which should have a `stackrc` which # Cells -Cells is a new scaling option with a full spec at http://wiki.openstack.org/blueprint-nova-compute-cells. +Cells is a new scaling option with a full spec at: +http://wiki.openstack.org/blueprint-nova-compute-cells. To setup a cells environment add the following to your `localrc` section: enable_service n-cell -Be aware that there are some features currently missing in cells, one notable one being security groups. The exercises have been patched to disable functionality not supported by cells. +Be aware that there are some features currently missing in cells, one notable +one being security groups. The exercises have been patched to disable +functionality not supported by cells. # Local Configuration -Historically DevStack has used ``localrc`` to contain all local configuration and customizations. More and more of the configuration variables available for DevStack are passed-through to the individual project configuration files. The old mechanism for this required specific code for each file and did not scale well. This is handled now by a master local configuration file. +Historically DevStack has used ``localrc`` to contain all local configuration +and customizations. More and more of the configuration variables available for +DevStack are passed-through to the individual project configuration files. +The old mechanism for this required specific code for each file and did not +scale well. This is handled now by a master local configuration file. # local.conf -The new config file ``local.conf`` is an extended-INI format that introduces a new meta-section header that provides some additional information such as a phase name and destination config filename: +The new config file ``local.conf`` is an extended-INI format that introduces +a new meta-section header that provides some additional information such +as a phase name and destination config filename: [[ | ]] @@ -339,11 +370,15 @@ If the path of the config file does not exist it is skipped. The defined phases are: * **local** - extracts ``localrc`` from ``local.conf`` before ``stackrc`` is sourced -* **post-config** - runs after the layer 2 services are configured and before they are started -* **extra** - runs after services are started and before any files in ``extra.d`` are executed +* **post-config** - runs after the layer 2 services are configured + and before they are started +* **extra** - runs after services are started and before any files + in ``extra.d`` are executed * **post-extra** - runs after files in ``extra.d`` are executed -The file is processed strictly in sequence; meta-sections may be specified more than once but if any settings are duplicated the last to appear in the file will be used. +The file is processed strictly in sequence; meta-sections may be specified more +than once but if any settings are duplicated the last to appear in the file +will be used. [[post-config|$NOVA_CONF]] [DEFAULT] diff --git a/clean.sh b/clean.sh index 3707d8411e..7851da3227 100755 --- a/clean.sh +++ b/clean.sh @@ -102,7 +102,7 @@ if is_service_enabled nova && [[ -r $NOVA_PLUGINS/hypervisor-$VIRT_DRIVER ]]; th fi # Clean out /etc -sudo rm -rf /etc/keystone /etc/glance /etc/nova /etc/cinder /etc/swift +sudo rm -rf /etc/keystone /etc/glance /etc/nova /etc/cinder /etc/swift /etc/heat /etc/neutron # Clean out tgt sudo rm -f /etc/tgt/conf.d/* diff --git a/driver_certs/cinder_driver_cert.sh b/driver_certs/cinder_driver_cert.sh index d2c636f89d..7726e7eb01 100755 --- a/driver_certs/cinder_driver_cert.sh +++ b/driver_certs/cinder_driver_cert.sh @@ -19,6 +19,8 @@ # # SCREEN_LOGDIR=/opt/stack/screen-logs +set -o pipefail + CERT_DIR=$(cd $(dirname "$0") && pwd) TOP_DIR=$(cd $CERT_DIR/..; pwd) diff --git a/exercises/client-env.sh b/exercises/client-env.sh index d955e4d1e1..cc518d9a06 100755 --- a/exercises/client-env.sh +++ b/exercises/client-env.sh @@ -28,7 +28,7 @@ TOP_DIR=$(cd $EXERCISE_DIR/..; pwd) source $TOP_DIR/functions # Import configuration -source $TOP_DIR/openrc +source $TOP_DIR/openrc admin # Import exercise configuration source $TOP_DIR/exerciserc @@ -64,7 +64,7 @@ if [[ "$ENABLED_SERVICES" =~ "key" ]]; then STATUS_KEYSTONE="Skipped" else echo -e "\nTest Keystone" - if keystone catalog --service identity; then + if openstack endpoint show identity; then STATUS_KEYSTONE="Succeeded" else STATUS_KEYSTONE="Failed" diff --git a/exercises/neutron-adv-test.sh b/exercises/neutron-adv-test.sh index 0a24fe9df7..6679670aee 100755 --- a/exercises/neutron-adv-test.sh +++ b/exercises/neutron-adv-test.sh @@ -100,15 +100,6 @@ PUBLIC_ROUTER1_NET="admin-net1" DEMO1_ROUTER1_NET="demo1-net1" DEMO2_ROUTER1_NET="demo2-net1" -KEYSTONE="keystone" - -# Manually create a token by querying keystone (sending JSON data). Keystone -# returns a token and catalog of endpoints. We use python to parse the token -# and save it. - -TOKEN=`keystone token-get | grep ' id ' | awk '{print $4}'` -die_if_not_set $LINENO TOKEN "Keystone fail to get token" - # Various functions # ----------------- @@ -150,21 +141,21 @@ function get_image_id { function get_tenant_id { local TENANT_NAME=$1 - local TENANT_ID=`keystone tenant-list | grep " $TENANT_NAME " | head -n 1 | get_field 1` + local TENANT_ID=`openstack project list | grep " $TENANT_NAME " | head -n 1 | get_field 1` die_if_not_set $LINENO TENANT_ID "Failure retrieving TENANT_ID for $TENANT_NAME" echo "$TENANT_ID" } function get_user_id { local USER_NAME=$1 - local USER_ID=`keystone user-list | grep $USER_NAME | awk '{print $2}'` + local USER_ID=`openstack user list | grep $USER_NAME | awk '{print $2}'` die_if_not_set $LINENO USER_ID "Failure retrieving USER_ID for $USER_NAME" echo "$USER_ID" } function get_role_id { local ROLE_NAME=$1 - local ROLE_ID=`keystone role-list | grep $ROLE_NAME | awk '{print $2}'` + local ROLE_ID=`openstack role list | grep $ROLE_NAME | awk '{print $2}'` die_if_not_set $LINENO ROLE_ID "Failure retrieving ROLE_ID for $ROLE_NAME" echo "$ROLE_ID" } @@ -199,28 +190,21 @@ function neutron_debug_admin { } function add_tenant { - local TENANT=$1 - local USER=$2 - - $KEYSTONE tenant-create --name=$TENANT - $KEYSTONE user-create --name=$USER --pass=${ADMIN_PASSWORD} - - local USER_ID=$(get_user_id $USER) - local TENANT_ID=$(get_tenant_id $TENANT) - - $KEYSTONE user-role-add --user-id $USER_ID --role-id $(get_role_id Member) --tenant-id $TENANT_ID + openstack project create $1 + openstack user create $2 --password ${ADMIN_PASSWORD} --project $1 + openstack role add Member --project $1 --user $2 } function remove_tenant { local TENANT=$1 local TENANT_ID=$(get_tenant_id $TENANT) - $KEYSTONE tenant-delete $TENANT_ID + openstack project delete $TENANT_ID } function remove_user { local USER=$1 local USER_ID=$(get_user_id $USER) - $KEYSTONE user-delete $USER_ID + openstack user delete $USER_ID } function create_tenants { diff --git a/extras.d/80-opendaylight.sh b/extras.d/80-opendaylight.sh index 57b43288e0..bfbabc2570 100644 --- a/extras.d/80-opendaylight.sh +++ b/extras.d/80-opendaylight.sh @@ -14,6 +14,7 @@ if is_service_enabled odl-server; then configure_opendaylight init_opendaylight elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then + configure_ml2_odl # This has to start before Neutron start_opendaylight elif [[ "$1" == "stack" && "$2" == "post-extra" ]]; then @@ -46,7 +47,11 @@ if is_service_enabled odl-compute; then ODL_MGR_PORT=${ODL_MGR_PORT:-6640} read ovstbl <<< $(sudo ovs-vsctl get Open_vSwitch . _uuid) sudo ovs-vsctl set-manager tcp:$ODL_MGR_IP:$ODL_MGR_PORT - sudo ovs-vsctl set Open_vSwitch $ovstbl other_config={"local_ip"="$ODL_LOCAL_IP"} + if [[ -n "$ODL_PROVIDER_MAPPINGS" ]] && [[ "$ENABLE_TENANT_VLANS" == "True" ]]; then + sudo ovs-vsctl set Open_vSwitch $ovstbl \ + other_config:provider_mappings=$ODL_PROVIDER_MAPPINGS + fi + sudo ovs-vsctl set Open_vSwitch $ovstbl other_config:local_ip=$ODL_LOCAL_IP elif [[ "$1" == "stack" && "$2" == "post-extra" ]]; then # no-op : diff --git a/files/apts/baremetal b/files/apts/baremetal index 54e76e0064..06ffab650c 100644 --- a/files/apts/baremetal +++ b/files/apts/baremetal @@ -1,6 +1,5 @@ busybox dnsmasq -gcc ipmitool make open-iscsi diff --git a/files/apts/cinder b/files/apts/cinder index 712fee99ec..7819c31655 100644 --- a/files/apts/cinder +++ b/files/apts/cinder @@ -2,6 +2,5 @@ tgt lvm2 qemu-utils libpq-dev -python-dev open-iscsi open-iscsi-utils # Deprecated since quantal dist:precise diff --git a/files/apts/general b/files/apts/general index 995c0c6f88..d81ec7a553 100644 --- a/files/apts/general +++ b/files/apts/general @@ -5,6 +5,7 @@ screen unzip wget psmisc +gcc git lsof # useful when debugging openssh-server @@ -18,5 +19,6 @@ tcpdump euca2ools # only for testing client tar python-cmd2 # dist:precise +python-dev python2.7 bc diff --git a/files/apts/glance b/files/apts/glance index 6dc878e4de..b5d8c77094 100644 --- a/files/apts/glance +++ b/files/apts/glance @@ -1,11 +1,9 @@ -gcc libffi-dev libmysqlclient-dev # testonly libpq-dev # testonly libssl-dev # testonly libxml2-dev libxslt1-dev # testonly -python-dev python-eventlet python-routes python-greenlet diff --git a/files/apts/ironic b/files/apts/ironic index a749ad762e..fe9c07f9d0 100644 --- a/files/apts/ironic +++ b/files/apts/ironic @@ -1,3 +1,5 @@ +ipmitool +iptables libguestfs0 libvirt-bin openssh-client diff --git a/files/apts/keystone b/files/apts/keystone index 564921b78b..57fde80aea 100644 --- a/files/apts/keystone +++ b/files/apts/keystone @@ -1,4 +1,3 @@ -python-dev python-lxml python-pastescript python-pastedeploy diff --git a/files/apts/n-api b/files/apts/n-api index e0e5e7f5b3..b4372d9361 100644 --- a/files/apts/n-api +++ b/files/apts/n-api @@ -1,2 +1,3 @@ python-dateutil msgpack-python +fping diff --git a/files/apts/nova b/files/apts/nova index 60119adeea..38c99c735b 100644 --- a/files/apts/nova +++ b/files/apts/nova @@ -15,6 +15,7 @@ sudo qemu-kvm # NOPRIME qemu # dist:wheezy,jessie NOPRIME libvirt-bin # NOPRIME +pm-utils libjs-jquery-tablesorter # Needed for coverage html reports vlan curl @@ -25,7 +26,6 @@ socat # used by ajaxterm python-mox python-paste python-migrate -python-gflags python-greenlet python-libvirt # NOPRIME python-libxml2 @@ -34,7 +34,6 @@ python-numpy # used by websockify for spice console python-pastedeploy python-eventlet python-cheetah -python-carrot python-tempita python-sqlalchemy python-suds diff --git a/files/apts/swift b/files/apts/swift index 37d5bc049e..080ecdb255 100644 --- a/files/apts/swift +++ b/files/apts/swift @@ -1,10 +1,8 @@ curl -gcc libffi-dev memcached python-configobj python-coverage -python-dev python-eventlet python-greenlet python-netifaces diff --git a/files/apts/trema b/files/apts/trema index 09cb7c6794..f685ca53b4 100644 --- a/files/apts/trema +++ b/files/apts/trema @@ -1,5 +1,4 @@ # Trema -gcc make ruby1.8 rubygems1.8 diff --git a/files/patches/unittest2-discover.patch b/files/patches/unittest2-discover.patch new file mode 100644 index 0000000000..347300d172 --- /dev/null +++ b/files/patches/unittest2-discover.patch @@ -0,0 +1,16 @@ +diff -r b2efb7df637b discover.py +--- a/discover.py Thu Mar 24 00:31:02 2011 -0400 ++++ b/discover.py Thu Nov 28 12:02:19 2013 +0000 +@@ -82,7 +82,11 @@ + """ + testMethodPrefix = 'test' + sortTestMethodsUsing = cmp +- suiteClass = unittest.TestSuite ++ try: ++ import unittest2 ++ suiteClass = unittest2.TestSuite ++ except ImportError: ++ suiteClass = unittest.TestSuite + _top_level_dir = None + + def loadTestsFromTestCase(self, testCaseClass): diff --git a/files/rpms-suse/general b/files/rpms-suse/general index ff27a3aac7..82cb09d934 100644 --- a/files/rpms-suse/general +++ b/files/rpms-suse/general @@ -3,6 +3,7 @@ bridge-utils ca-certificates-mozilla curl euca2ools +gcc git-core iputils libopenssl-devel # to rebuild pyOpenSSL if needed diff --git a/files/rpms-suse/glance b/files/rpms-suse/glance index d9844e9bb4..edd1564e92 100644 --- a/files/rpms-suse/glance +++ b/files/rpms-suse/glance @@ -1,4 +1,3 @@ -gcc libxml2-devel python-PasteDeploy python-Routes diff --git a/files/rpms-suse/n-api b/files/rpms-suse/n-api index 0f08daace3..6f59e603b2 100644 --- a/files/rpms-suse/n-api +++ b/files/rpms-suse/n-api @@ -1 +1,2 @@ python-dateutil +fping diff --git a/files/rpms-suse/neutron b/files/rpms-suse/neutron index e9ccf5921b..d4841b161a 100644 --- a/files/rpms-suse/neutron +++ b/files/rpms-suse/neutron @@ -1,9 +1,9 @@ dnsmasq -dnsmasq-utils # dist:opensuse-12.3 +dnsmasq-utils # dist:opensuse-12.3,opensuse-13.1 ebtables iptables iputils -mysql-community-server # NOPRIME +mariadb # NOPRIME python-boto python-eventlet python-greenlet diff --git a/files/rpms-suse/nova b/files/rpms-suse/nova index ee4917d702..3e95724150 100644 --- a/files/rpms-suse/nova +++ b/files/rpms-suse/nova @@ -1,6 +1,6 @@ curl dnsmasq -dnsmasq-utils # dist:opensuse-12.3 +dnsmasq-utils # dist:opensuse-12.3,opensuse-13.1 ebtables gawk genisoimage # required for config_drive @@ -12,8 +12,7 @@ kvm # NOPRIME qemu # NOPRIME libvirt # NOPRIME libvirt-python # NOPRIME -libxml2-python -mysql-community-server # NOPRIME +mariadb # NOPRIME parted polkit python-M2Crypto @@ -24,20 +23,19 @@ python-Routes python-SQLAlchemy python-Tempita python-boto -python-carrot python-cheetah python-eventlet python-feedparser python-greenlet python-iso8601 python-kombu +python-libxml2 python-lockfile python-lxml # needed for glance which is needed for nova --- this shouldn't be here python-mox python-mysql python-numpy # needed by websockify for spice console python-paramiko -python-python-gflags python-sqlalchemy-migrate python-suds python-xattr # needed for glance which is needed for nova --- this shouldn't be here diff --git a/files/rpms-suse/opendaylight b/files/rpms-suse/opendaylight index d6c7146331..f7fafffae1 100644 --- a/files/rpms-suse/opendaylight +++ b/files/rpms-suse/opendaylight @@ -1,4 +1,3 @@ openvswitch # NOPRIME -openvswitch-controller # NOPRIME openvswitch-switch # NOPRIME diff --git a/files/rpms-suse/swift b/files/rpms-suse/swift index f3c95aad98..4b14098064 100644 --- a/files/rpms-suse/swift +++ b/files/rpms-suse/swift @@ -1,5 +1,4 @@ curl -gcc memcached python-PasteDeploy python-WebOb diff --git a/files/rpms/cinder b/files/rpms/cinder index 423d57cd98..ce6181eedf 100644 --- a/files/rpms/cinder +++ b/files/rpms/cinder @@ -1,7 +1,6 @@ lvm2 scsi-target-utils qemu-img -python-devel postgresql-devel iscsi-initiator-utils python-lxml #dist:f19,f20,rhel7 diff --git a/files/rpms/general b/files/rpms/general index 6cfe31eaf1..c940de6dfe 100644 --- a/files/rpms/general +++ b/files/rpms/general @@ -2,19 +2,20 @@ bridge-utils curl dbus euca2ools # only for testing client -gcc # dist:rhel6 [2] +gcc git-core openssh-server openssl openssl-devel # to rebuild pyOpenSSL if needed -libxml2-devel # dist:rhel6 [2] -libxslt-devel # dist:rhel6 [2] +libxml2-devel +libxslt-devel psmisc pylint python-setuptools python-prettytable # dist:rhel6 [1] python-unittest2 python-virtualenv +python-devel screen tar tcpdump @@ -27,7 +28,3 @@ bc # but others have versioned (<=0.7). So if a later version (0.7.1) # gets installed in response to an unversioned dependency, it breaks. # This pre-installs a compatible 0.6(ish) version from RHEL - -# [2] : RHEL6 rpm versions of python-lxml is old, and has to be -# removed. Several tools rely on it, so we install the dependencies -# pip needs to build it here (see tools/install_prereqs.sh) diff --git a/files/rpms/glance b/files/rpms/glance index 2007e2ea9e..fc07fa787b 100644 --- a/files/rpms/glance +++ b/files/rpms/glance @@ -1,4 +1,3 @@ -gcc libffi-devel libxml2-devel # testonly libxslt-devel # testonly @@ -6,7 +5,6 @@ mysql-devel # testonly openssl-devel # testonly postgresql-devel # testonly python-argparse -python-devel python-eventlet python-greenlet python-lxml #dist:f19,f20,rhel7 diff --git a/files/rpms/horizon b/files/rpms/horizon index 2dd24e0763..92afed2985 100644 --- a/files/rpms/horizon +++ b/files/rpms/horizon @@ -1,6 +1,4 @@ Django -django-registration -gcc httpd # NOPRIME mod_wsgi # NOPRIME pylint diff --git a/files/rpms/ironic b/files/rpms/ironic index 54b98299ee..0c81081648 100644 --- a/files/rpms/ironic +++ b/files/rpms/ironic @@ -1,6 +1,9 @@ +ipmitool +iptables libguestfs libvirt libvirt-python +net-tools openssh-clients openvswitch python-libguestfs diff --git a/files/rpms/n-api b/files/rpms/n-api index 0f08daace3..6f59e603b2 100644 --- a/files/rpms/n-api +++ b/files/rpms/n-api @@ -1 +1,2 @@ python-dateutil +fping diff --git a/files/rpms/neutron b/files/rpms/neutron index 06ea0ea35d..9fafecbf52 100644 --- a/files/rpms/neutron +++ b/files/rpms/neutron @@ -21,4 +21,3 @@ rabbitmq-server # NOPRIME qpid-cpp-server # NOPRIME sqlite sudo -vconfig diff --git a/files/rpms/nova b/files/rpms/nova index 45d6e0bfb1..e05d0d7a7e 100644 --- a/files/rpms/nova +++ b/files/rpms/nova @@ -17,11 +17,9 @@ mysql-server # NOPRIME parted polkit python-boto -python-carrot python-cheetah python-eventlet python-feedparser -python-gflags python-greenlet python-iso8601 python-kombu @@ -42,4 +40,3 @@ rabbitmq-server # NOPRIME qpid-cpp-server # NOPRIME sqlite sudo -vconfig diff --git a/files/rpms/swift b/files/rpms/swift index bf29ea29b7..938d2c8fe6 100644 --- a/files/rpms/swift +++ b/files/rpms/swift @@ -1,10 +1,8 @@ curl -gcc libffi-devel memcached python-configobj python-coverage -python-devel python-eventlet python-greenlet python-netifaces diff --git a/functions-common b/functions-common index e6caaa3bff..613a86cd84 100644 --- a/functions-common +++ b/functions-common @@ -881,29 +881,43 @@ function get_packages { # Distro-agnostic package installer # install_package package [package ...] -function install_package { - local xtrace=$(set +o | grep xtrace) - set +o xtrace - if is_ubuntu; then - # if there are transient errors pulling the updates, that's fine. It may - # be secondary repositories that we don't really care about. - [[ "$NO_UPDATE_REPOS" = "True" ]] || apt_get update || /bin/true - NO_UPDATE_REPOS=True +function update_package_repo { + if [[ "NO_UPDATE_REPOS" = "True" ]]; then + return 0 + fi + if is_ubuntu; then + local xtrace=$(set +o | grep xtrace) + set +o xtrace + if [[ "$REPOS_UPDATED" != "True" || "$RETRY_UPDATE" = "True" ]]; then + # if there are transient errors pulling the updates, that's fine. + # It may be secondary repositories that we don't really care about. + apt_get update || /bin/true + REPOS_UPDATED=True + fi $xtrace + fi +} + +function real_install_package { + if is_ubuntu; then apt_get install "$@" elif is_fedora; then - $xtrace yum_install "$@" elif is_suse; then - $xtrace zypper_install "$@" else - $xtrace exit_distro_not_supported "installing packages" fi } +# Distro-agnostic package installer +# install_package package [package ...] +function install_package { + update_package_repo + real_install_package $@ || RETRY_UPDATE=True update_package_repo && real_install_package $@ +} + # Distro-agnostic function to tell if a package is installed # is_package_installed package [package ...] function is_package_installed { @@ -1067,7 +1081,7 @@ function screen_it { # sleep to allow bash to be ready to be send the command - we are # creating a new window in screen and then sends characters, so if # bash isn't running by the time we send the command, nothing happens - sleep 1.5 + sleep 3 NL=`echo -ne '\015'` # This fun command does the following: @@ -1231,14 +1245,27 @@ function pip_install { $xtrace $SUDO_PIP PIP_DOWNLOAD_CACHE=${PIP_DOWNLOAD_CACHE:-/var/cache/pip} \ - HTTP_PROXY=$http_proxy \ - HTTPS_PROXY=$https_proxy \ - NO_PROXY=$no_proxy \ + http_proxy=$http_proxy \ + https_proxy=$https_proxy \ + no_proxy=$no_proxy \ $CMD_PIP install --build=${pip_build_tmp} \ $PIP_MIRROR_OPT $@ \ && $SUDO_PIP rm -rf ${pip_build_tmp} } +# this should be used if you want to install globally, all libraries should +# use this, especially *oslo* ones +function setup_install { + local project_dir=$1 + setup_package_with_req_sync $project_dir +} + +# this should be used for projects which run services, like all services +function setup_develop { + local project_dir=$1 + setup_package_with_req_sync $project_dir -e +} + # ``pip install -e`` the package, which processes the dependencies # using pip before running `setup.py develop` # @@ -1247,8 +1274,9 @@ function pip_install { # # Uses globals ``TRACK_DEPENDS``, ``REQUIREMENTS_DIR``, ``UNDO_REQUIREMENTS`` # setup_develop directory -function setup_develop { +function setup_package_with_req_sync { local project_dir=$1 + local flags=$2 # Don't update repo if local changes exist # Don't use buggy "git diff --quiet" @@ -1260,7 +1288,7 @@ function setup_develop { $SUDO_CMD python update.py $project_dir) fi - setup_develop_no_requirements_update $project_dir + setup_package $project_dir $flags # We've just gone and possibly modified the user's source tree in an # automated way, which is considered bad form if it's a development @@ -1281,12 +1309,15 @@ function setup_develop { # using pip before running `setup.py develop` # Uses globals ``STACK_USER`` # setup_develop_no_requirements_update directory -function setup_develop_no_requirements_update { +function setup_package { local project_dir=$1 + local flags=$2 - pip_install -e $project_dir + pip_install $flags $project_dir # ensure that further actions can do things like setup.py sdist - safe_chown -R $STACK_USER $1/*.egg-info + if [[ "$flags" == "-e" ]]; then + safe_chown -R $STACK_USER $1/*.egg-info + fi } diff --git a/lib/apache b/lib/apache index 2d5e39a65d..baf0fbc5f5 100644 --- a/lib/apache +++ b/lib/apache @@ -11,6 +11,7 @@ # - is_apache_enabled_service # - install_apache_wsgi # - config_apache_wsgi +# - apache_site_config_for # - enable_apache_site # - disable_apache_site # - start_apache_server @@ -30,13 +31,13 @@ APACHE_GROUP=${APACHE_GROUP:-$(id -gn $APACHE_USER)} # Set up apache name and configuration directory if is_ubuntu; then APACHE_NAME=apache2 - APACHE_CONF_DIR=sites-available + APACHE_CONF_DIR=${APACHE_CONF_DIR:-/etc/$APACHE_NAME/sites-available} elif is_fedora; then APACHE_NAME=httpd - APACHE_CONF_DIR=conf.d + APACHE_CONF_DIR=${APACHE_CONF_DIR:-/etc/$APACHE_NAME/conf.d} elif is_suse; then APACHE_NAME=apache2 - APACHE_CONF_DIR=vhosts.d + APACHE_CONF_DIR=${APACHE_CONF_DIR:-/etc/$APACHE_NAME/vhosts.d} fi # Functions @@ -78,14 +79,62 @@ function install_apache_wsgi { fi } +# apache_site_config_for() - The filename of the site's configuration file. +# This function uses the global variables APACHE_NAME and APACHE_CONF_DIR. +# +# On Ubuntu 14.04, the site configuration file must have a .conf suffix for a2ensite and a2dissite to +# recognise it. a2ensite and a2dissite ignore the .conf suffix used as parameter. The default sites' +# files are 000-default.conf and default-ssl.conf. +# +# On Ubuntu 12.04, the site configuration file may have any format, as long as it is in +# /etc/apache2/sites-available/. a2ensite and a2dissite need the entire file name to work. The default +# sites' files are default and default-ssl. +# +# On Fedora, any file in /etc/httpd/conf.d/ whose name ends with .conf is enabled. +# +# On RHEL and CentOS, things should hopefully work as in Fedora. +# +# The table below summarizes what should happen on each distribution: +# +----------------------+--------------------+--------------------------+--------------------------+ +# | Distribution | File name | Site enabling command | Site disabling command | +# +----------------------+--------------------+--------------------------+--------------------------+ +# | Ubuntu 12.04 | site | a2ensite site | a2dissite site | +# | Ubuntu 14.04 | site.conf | a2ensite site | a2dissite site | +# | Fedora, RHEL, CentOS | site.conf.disabled | mv site.conf{.disabled,} | mv site.conf{,.disabled} | +# +----------------------+--------------------+--------------------------+--------------------------+ +function apache_site_config_for { + local site=$@ + if is_ubuntu; then + local apache_version=$(sudo /usr/sbin/apache2ctl -v | awk '/Server version/ {print $3}' | cut -f2 -d/) + if [[ "$apache_version" =~ ^2\.2\. ]]; then + # Ubuntu 12.04 - Apache 2.2 + echo $APACHE_CONF_DIR/${site} + else + # Ubuntu 14.04 - Apache 2.4 + echo $APACHE_CONF_DIR/${site}.conf + fi + elif is_fedora; then + # fedora conf.d is only imported if it ends with .conf so this is approx the same + local enabled_site_file="$APACHE_CONF_DIR/${site}.conf" + if [ -f $enabled_site_file ]; then + echo ${enabled_site_file} + else + echo ${enabled_site_file}.disabled + fi + fi +} + # enable_apache_site() - Enable a particular apache site function enable_apache_site { local site=$@ if is_ubuntu; then sudo a2ensite ${site} elif is_fedora; then - # fedora conf.d is only imported if it ends with .conf so this is approx the same - sudo mv /etc/$APACHE_NAME/$APACHE_CONF_DIR/${site} /etc/$APACHE_NAME/$APACHE_CONF_DIR/${site}.conf + local enabled_site_file="$APACHE_CONF_DIR/${site}.conf" + # Do nothing if site already enabled or no site config exists + if [[ -f ${enabled_site_file}.disabled ]] && [[ ! -f ${enabled_site_file} ]]; then + sudo mv ${enabled_site_file}.disabled ${enabled_site_file} + fi fi } @@ -95,7 +144,11 @@ function disable_apache_site { if is_ubuntu; then sudo a2dissite ${site} elif is_fedora; then - sudo mv /etc/$APACHE_NAME/$APACHE_CONF_DIR/${site}.conf /etc/$APACHE_NAME/$APACHE_CONF_DIR/${site} + local enabled_site_file="$APACHE_CONF_DIR/${site}.conf" + # Do nothing if no site config exists + if [[ -f ${enabled_site_file} ]]; then + sudo mv ${enabled_site_file} ${enabled_site_file}.disabled + fi fi } diff --git a/lib/ceilometer b/lib/ceilometer index 5030b3cb35..a4be7af480 100644 --- a/lib/ceilometer +++ b/lib/ceilometer @@ -147,18 +147,22 @@ function configure_ceilometer { # Install the policy file for the API server cp $CEILOMETER_DIR/etc/ceilometer/policy.json $CEILOMETER_CONF_DIR - cp $CEILOMETER_DIR/etc/ceilometer/pipeline.yaml $CEILOMETER_CONF_DIR iniset $CEILOMETER_CONF DEFAULT policy_file $CEILOMETER_CONF_DIR/policy.json + cp $CEILOMETER_DIR/etc/ceilometer/pipeline.yaml $CEILOMETER_CONF_DIR + cp $CEILOMETER_DIR/etc/ceilometer/api_paste.ini $CEILOMETER_CONF_DIR + cp $CEILOMETER_DIR/etc/ceilometer/event_definitions.yaml $CEILOMETER_CONF_DIR + if [ "$CEILOMETER_PIPELINE_INTERVAL" ]; then sed -i "s/interval:.*/interval: ${CEILOMETER_PIPELINE_INTERVAL}/" $CEILOMETER_CONF_DIR/pipeline.yaml fi # the compute and central agents need these credentials in order to - # call out to the public nova and glance APIs - iniset $CEILOMETER_CONF DEFAULT os_username ceilometer - iniset $CEILOMETER_CONF DEFAULT os_password $SERVICE_PASSWORD - iniset $CEILOMETER_CONF DEFAULT os_tenant_name $SERVICE_TENANT_NAME + # call out to other services' public APIs + # the alarm evaluator needs these options to call ceilometer APIs + iniset $CEILOMETER_CONF service_credentials os_username ceilometer + iniset $CEILOMETER_CONF service_credentials os_password $SERVICE_PASSWORD + iniset $CEILOMETER_CONF service_credentials os_tenant_name $SERVICE_TENANT_NAME iniset $CEILOMETER_CONF keystone_authtoken auth_host $KEYSTONE_AUTH_HOST iniset $CEILOMETER_CONF keystone_authtoken auth_port $KEYSTONE_AUTH_PORT @@ -236,7 +240,7 @@ function install_ceilometerclient { # start_ceilometer() - Start running processes, including screen function start_ceilometer { if [[ "$VIRT_DRIVER" = 'libvirt' ]]; then - screen_it ceilometer-acompute "cd ; sg $LIBVIRT_GROUP \"ceilometer-agent-compute --config-file $CEILOMETER_CONF\"" + screen_it ceilometer-acompute "cd ; sg $LIBVIRT_GROUP 'ceilometer-agent-compute --config-file $CEILOMETER_CONF'" fi if [[ "$VIRT_DRIVER" = 'vsphere' ]]; then screen_it ceilometer-acompute "cd ; ceilometer-agent-compute --config-file $CEILOMETER_CONF" diff --git a/lib/cinder b/lib/cinder index dd2956a5b4..d5ee17e65b 100644 --- a/lib/cinder +++ b/lib/cinder @@ -233,8 +233,6 @@ function configure_cinder { inicomment $CINDER_API_PASTE_INI filter:authtoken admin_password inicomment $CINDER_API_PASTE_INI filter:authtoken signing_dir - cp $CINDER_DIR/etc/cinder/cinder.conf.sample $CINDER_CONF - iniset $CINDER_CONF keystone_authtoken auth_host $KEYSTONE_AUTH_HOST iniset $CINDER_CONF keystone_authtoken auth_port $KEYSTONE_AUTH_PORT iniset $CINDER_CONF keystone_authtoken auth_protocol $KEYSTONE_AUTH_PROTOCOL @@ -274,6 +272,10 @@ function configure_cinder { iniset $CINDER_CONF DEFAULT lock_path $CINDER_STATE_PATH iniset $CINDER_CONF DEFAULT periodic_interval $CINDER_PERIODIC_INTERVAL + if is_service_enabled swift; then + iniset $CINDER_CONF DEFAULT backup_swift_url "http://$SERVICE_HOST:8080/v1/AUTH_" + fi + if is_service_enabled ceilometer; then iniset $CINDER_CONF DEFAULT notification_driver "cinder.openstack.common.notifier.rpc_notifier" fi diff --git a/lib/cinder_plugins/nfs b/lib/cinder_plugins/nfs index 2d9d875245..5f4cc5369a 100644 --- a/lib/cinder_plugins/nfs +++ b/lib/cinder_plugins/nfs @@ -30,8 +30,7 @@ set +o xtrace function configure_cinder_driver { iniset $CINDER_CONF DEFAULT volume_driver "cinder.volume.drivers.nfs.NfsDriver" iniset $CINDER_CONF DEFAULT nfs_shares_config "$CINDER_CONF_DIR/nfs_shares.conf" - echo "$CINDER_NFS_SERVERPATH" | sudo tee "$CINDER_CONF_DIR/nfs_shares.conf" - sudo chmod 660 $CINDER_CONF_DIR/nfs_shares.conf + echo "$CINDER_NFS_SERVERPATH" | tee "$CINDER_CONF_DIR/nfs_shares.conf" } # Restore xtrace diff --git a/lib/databases/mysql b/lib/databases/mysql index 7a0145ae1b..0ccfce5454 100644 --- a/lib/databases/mysql +++ b/lib/databases/mysql @@ -47,6 +47,7 @@ function recreate_database_mysql { } function configure_database_mysql { + local slow_log echo_summary "Configuring and starting MySQL" if is_ubuntu; then @@ -83,36 +84,33 @@ function configure_database_mysql { # Now update ``my.cnf`` for some local needs and restart the mysql service - # Change ‘bind-address’ from localhost (127.0.0.1) to any (0.0.0.0) - sudo sed -i '/^bind-address/s/127.0.0.1/0.0.0.0/g' $MY_CONF + # Change ‘bind-address’ from localhost (127.0.0.1) to any (0.0.0.0) and + # set default db type to InnoDB + sudo bash -c "source $TOP_DIR/functions && \ + iniset $MY_CONF mysqld bind-address 0.0.0.0 && \ + iniset $MY_CONF mysqld sql_mode STRICT_ALL_TABLES && \ + iniset $MY_CONF mysqld default-storage-engine InnoDB" - # Set default db type to InnoDB - if sudo grep -q "default-storage-engine" $MY_CONF; then - # Change it - sudo bash -c "source $TOP_DIR/functions; iniset $MY_CONF mysqld default-storage-engine InnoDB" - else - # Add it - sudo sed -i -e "/^\[mysqld\]/ a \ -default-storage-engine = InnoDB" $MY_CONF - fi if [[ "$DATABASE_QUERY_LOGGING" == "True" ]]; then echo_summary "Enabling MySQL query logging" - - # Turn on slow query log - sudo sed -i '/log.slow.queries/d' $MY_CONF - sudo sed -i -e "/^\[mysqld\]/ a \ - log-slow-queries = /var/log/mysql/mysql-slow.log" $MY_CONF - - # Log all queries (any query taking longer than 0 seconds) - sudo sed -i '/long.query.time/d' $MY_CONF - sudo sed -i -e "/^\[mysqld\]/ a \ - long-query-time = 0" $MY_CONF - - # Log all non-indexed queries - sudo sed -i '/log.queries.not.using.indexes/d' $MY_CONF - sudo sed -i -e "/^\[mysqld\]/ a \ - log-queries-not-using-indexes" $MY_CONF + if is_fedora && ! [[ $DISTRO =~ (rhel6) ]]; then + slow_log=/var/log/mariadb/mariadb-slow.log + else + slow_log=/var/log/mysql/mysql-slow.log + fi + sudo sed -e '/log.slow.queries/d' \ + -e '/long.query.time/d' \ + -e '/log.queries.not.using.indexes/d' \ + -i $MY_CONF + + # Turn on slow query log, log all queries (any query taking longer than + # 0 seconds) and log all non-indexed queries + sudo bash -c "source $TOP_DIR/functions && \ + iniset $MY_CONF mysqld slow-query-log 1 && \ + iniset $MY_CONF mysqld slow-query-log-file $slow_log && \ + iniset $MY_CONF mysqld long-query-time 0 && \ + iniset $MY_CONF mysqld log-queries-not-using-indexes 1" fi diff --git a/lib/heat b/lib/heat index f66f0a8454..fe75ec9dc4 100644 --- a/lib/heat +++ b/lib/heat @@ -96,7 +96,7 @@ function configure_heat { iniset $HEAT_CONF DEFAULT heat_waitcondition_server_url http://$HEAT_API_CFN_HOST:$HEAT_API_CFN_PORT/v1/waitcondition iniset $HEAT_CONF DEFAULT heat_watch_server_url http://$HEAT_API_CW_HOST:$HEAT_API_CW_PORT iniset $HEAT_CONF database connection `database_connection_url heat` - iniset $HEAT_CONF DEFAULT auth_encryption_key `hexdump -n 16 -v -e '/1 "%02x"' /dev/random` + iniset $HEAT_CONF DEFAULT auth_encryption_key `hexdump -n 16 -v -e '/1 "%02x"' /dev/urandom` # logging iniset $HEAT_CONF DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL @@ -119,7 +119,6 @@ function configure_heat { # ec2authtoken iniset $HEAT_CONF ec2authtoken auth_uri $KEYSTONE_SERVICE_PROTOCOL://$KEYSTONE_SERVICE_HOST:$KEYSTONE_SERVICE_PORT/v2.0 - iniset $HEAT_CONF ec2authtoken keystone_ec2_uri $KEYSTONE_SERVICE_PROTOCOL://$KEYSTONE_SERVICE_HOST:$KEYSTONE_SERVICE_PORT/v2.0/ec2tokens # paste_deploy [[ "$HEAT_STANDALONE" = "True" ]] && iniset $HEAT_CONF paste_deploy flavor standalone diff --git a/lib/horizon b/lib/horizon index 27c2d26a01..02715cedff 100644 --- a/lib/horizon +++ b/lib/horizon @@ -45,7 +45,10 @@ function _horizon_config_set { local option=$3 local value=$4 - if grep -q "^$section" $file; then + if [ -z "$section" ]; then + sed -e "/^$option/d" -i $local_settings + echo -e "\n$option=$value" >> $file + elif grep -q "^$section" $file; then line=$(sed -ne "/^$section/,/^}/ { /^ *'$option':/ p; }" $file) if [ -n "$line" ]; then sed -i -e "/^$section/,/^}/ s/^\( *'$option'\) *:.*$/\1: $value,/" $file @@ -72,6 +75,9 @@ function cleanup_horizon { sudo rm /usr/bin/node fi fi + + local horizon_conf=$(apache_site_config_for horizon) + sudo rm -f $horizon_conf } # configure_horizon() - Set config files, create data dirs, etc @@ -103,6 +109,13 @@ function init_horizon { _horizon_config_set $local_settings OPENSTACK_NEUTRON_NETWORK enable_vpn True fi + _horizon_config_set $local_settings "" OPENSTACK_HOST \"${KEYSTONE_SERVICE_HOST}\" + _horizon_config_set $local_settings "" OPENSTACK_KEYSTONE_URL "\"${KEYSTONE_SERVICE_PROTOCOL}://%s:${KEYSTONE_SERVICE_PORT}/v2.0\" % OPENSTACK_HOST" + + if [ -f $SSL_BUNDLE_FILE ]; then + _horizon_config_set $local_settings "" OPENSTACK_SSL_CACERT \"${SSL_BUNDLE_FILE}\" + fi + # Create an empty directory that apache uses as docroot sudo mkdir -p $HORIZON_DIR/.blackhole @@ -112,13 +125,21 @@ function init_horizon { HORIZON_REQUIRE='Require all granted' fi - local horizon_conf=/etc/$APACHE_NAME/$APACHE_CONF_DIR/horizon.conf + local horizon_conf=$(apache_site_config_for horizon) + + # Configure apache to run horizon + sudo sh -c "sed -e \" + s,%USER%,$APACHE_USER,g; + s,%GROUP%,$APACHE_GROUP,g; + s,%HORIZON_DIR%,$HORIZON_DIR,g; + s,%APACHE_NAME%,$APACHE_NAME,g; + s,%DEST%,$DEST,g; + s,%HORIZON_REQUIRE%,$HORIZON_REQUIRE,g; + \" $FILES/apache-horizon.template >$horizon_conf" + if is_ubuntu; then - # Clean up the old config name - sudo rm -f /etc/apache2/sites-enabled/000-default - # Be a good citizen and use the distro tools here + disable_apache_site 000-default sudo touch $horizon_conf - sudo a2ensite horizon.conf elif is_fedora; then sudo sed '/^Listen/s/^.*$/Listen 0.0.0.0:80/' -i /etc/httpd/conf/httpd.conf elif is_suse; then @@ -126,20 +147,12 @@ function init_horizon { else exit_distro_not_supported "horizon apache configuration" fi + enable_apache_site horizon # Remove old log files that could mess with how devstack detects whether Horizon # has been successfully started (see start_horizon() and functions::screen_it()) sudo rm -f /var/log/$APACHE_NAME/horizon_* - # Configure apache to run horizon - sudo sh -c "sed -e \" - s,%USER%,$APACHE_USER,g; - s,%GROUP%,$APACHE_GROUP,g; - s,%HORIZON_DIR%,$HORIZON_DIR,g; - s,%APACHE_NAME%,$APACHE_NAME,g; - s,%DEST%,$DEST,g; - s,%HORIZON_REQUIRE%,$HORIZON_REQUIRE,g; - \" $FILES/apache-horizon.template >$horizon_conf" } # install_horizon() - Collect source and prepare diff --git a/lib/infra b/lib/infra index 7f70ff2787..e2f7dadf3d 100644 --- a/lib/infra +++ b/lib/infra @@ -46,7 +46,7 @@ function install_infra { # Install pbr git_clone $PBR_REPO $PBR_DIR $PBR_BRANCH - setup_develop $PBR_DIR + setup_install $PBR_DIR } # Restore xtrace diff --git a/lib/ironic b/lib/ironic index 979420f2cb..0656980a89 100644 --- a/lib/ironic +++ b/lib/ironic @@ -42,28 +42,35 @@ IRONIC_POLICY_JSON=$IRONIC_CONF_DIR/policy.json IRONIC_SCRIPTS_DIR=${IRONIC_SCRIPTS_DIR:-$TOP_DIR/tools/ironic/scripts} IRONIC_TEMPLATES_DIR=${IRONIC_TEMPLATES_DIR:-$TOP_DIR/tools/ironic/templates} IRONIC_BAREMETAL_BASIC_OPS=$(trueorfalse False $IRONIC_BAREMETAL_BASIC_OPS) +IRONIC_ENABLED_DRIVERS=${IRONIC_ENABLED_DRIVERS:-fake,pxe_ssh,pxe_ipmitool} IRONIC_SSH_USERNAME=${IRONIC_SSH_USERNAME:-`whoami`} IRONIC_SSH_KEY_DIR=${IRONIC_SSH_KEY_DIR:-$IRONIC_DATA_DIR/ssh_keys} IRONIC_SSH_KEY_FILENAME=${IRONIC_SSH_KEY_FILENAME:-ironic_key} IRONIC_KEY_FILE=$IRONIC_SSH_KEY_DIR/$IRONIC_SSH_KEY_FILENAME IRONIC_SSH_VIRT_TYPE=${IRONIC_SSH_VIRT_TYPE:-virsh} IRONIC_TFTPBOOT_DIR=${IRONIC_TFTPBOOT_DIR:-$IRONIC_DATA_DIR/tftpboot} -IRONIC_VM_SSH_PORT=${IRONIC_VM_SSH_PORT:-2222} +IRONIC_VM_SSH_PORT=${IRONIC_VM_SSH_PORT:-22} IRONIC_VM_SSH_ADDRESS=${IRONIC_VM_SSH_ADDRESS:-$HOST_IP} IRONIC_VM_COUNT=${IRONIC_VM_COUNT:-1} IRONIC_VM_SPECS_CPU=${IRONIC_VM_SPECS_CPU:-1} -# NOTE(agordeev): both ubuntu and fedora deploy images won't work with 256MB of RAM. -# System halts and throws kernel panic during initramfs unpacking. -# Ubuntu needs at least 384MB, but fedora requires 448. -# So placing 512 here to satisfy both. -IRONIC_VM_SPECS_RAM=${IRONIC_VM_SPECS_RAM:-512} +# NOTE(adam_g): Kernels 3.12 and newer user tmpfs by default for initramfs. +# DIB produced ramdisks tend to be ~250MB but tmpfs will only allow +# use of 50% of available memory before ENOSPC. Set minimum 1GB +# for nodes to avoid (LP: #1311987) and ensure consistency across +# older and newer kernels. +IRONIC_VM_SPECS_RAM=${IRONIC_VM_SPECS_RAM:-1024} IRONIC_VM_SPECS_DISK=${IRONIC_VM_SPECS_DISK:-10} +IRONIC_VM_EPHEMERAL_DISK=${IRONIC_VM_EPHEMERAL_DISK:-0} IRONIC_VM_EMULATOR=${IRONIC_VM_EMULATOR:-/usr/bin/qemu-system-x86_64} IRONIC_VM_NETWORK_BRIDGE=${IRONIC_VM_NETWORK_BRIDGE:-brbm} IRONIC_VM_NETWORK_RANGE=${IRONIC_VM_NETWORK_RANGE:-192.0.2.0/24} IRONIC_VM_MACS_CSV_FILE=${IRONIC_VM_MACS_CSV_FILE:-$IRONIC_DATA_DIR/ironic_macs.csv} IRONIC_AUTHORIZED_KEYS_FILE=${IRONIC_AUTHORIZED_KEYS_FILE:-$HOME/.ssh/authorized_keys} +# By default, baremetal VMs will console output to file. +IRONIC_VM_LOG_CONSOLE=${IRONIC_VM_LOG_CONSOLE:-True} +IRONIC_VM_LOG_DIR=${IRONIC_VM_LOG_DIR:-$IRONIC_DATA_DIR/logs/} + DIB_DIR=${DIB_DIR:-$DEST/diskimage-builder} # Use DIB to create deploy ramdisk and kernel. @@ -140,6 +147,11 @@ function configure_ironic { configure_ironic_api fi + # Format logging + if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ]; then + setup_colorized_logging $IRONIC_CONF_FILE DEFAULT + fi + if [[ "$IRONIC_BAREMETAL_BASIC_OPS" == "True" ]]; then configure_ironic_auxiliary fi @@ -169,12 +181,25 @@ function configure_ironic_api { function configure_ironic_conductor { cp $IRONIC_DIR/etc/ironic/rootwrap.conf $IRONIC_ROOTWRAP_CONF cp -r $IRONIC_DIR/etc/ironic/rootwrap.d $IRONIC_CONF_DIR + IRONIC_ROOTWRAP=$(get_rootwrap_location ironic) + ROOTWRAP_ISUDOER_CMD="$IRONIC_ROOTWRAP $IRONIC_CONF_DIR/rootwrap.conf *" + + # Set up the rootwrap sudoers for ironic + TEMPFILE=`mktemp` + echo "$STACK_USER ALL=(root) NOPASSWD: $ROOTWRAP_ISUDOER_CMD" >$TEMPFILE + chmod 0440 $TEMPFILE + sudo chown root:root $TEMPFILE + sudo mv $TEMPFILE /etc/sudoers.d/ironic-rootwrap iniset $IRONIC_CONF_FILE DEFAULT rootwrap_config $IRONIC_ROOTWRAP_CONF + iniset $IRONIC_CONF_FILE DEFAULT enabled_drivers $IRONIC_ENABLED_DRIVERS iniset $IRONIC_CONF_FILE conductor api_url http://$HOST_IP:6385 iniset $IRONIC_CONF_FILE pxe tftp_server $HOST_IP iniset $IRONIC_CONF_FILE pxe tftp_root $IRONIC_TFTPBOOT_DIR iniset $IRONIC_CONF_FILE pxe tftp_master_path $IRONIC_TFTPBOOT_DIR/master_images + if [[ "$IRONIC_VM_LOG_CONSOLE" == "True" ]] ; then + iniset $IRONIC_CONF_FILE pxe pxe_append_params "nofb nomodeset vga=normal console=ttyS0" + fi } # create_ironic_cache_dir() - Part of the init_ironic() process @@ -304,9 +329,15 @@ function configure_ironic_dirs { function create_bridge_and_vms { # Call libvirt setup scripts in a new shell to ensure any new group membership sudo su $STACK_USER -c "$IRONIC_SCRIPTS_DIR/setup-network" + if [[ "$IRONIC_VM_LOG_CONSOLE" == "True" ]] ; then + LOG_ARG="$IRONIC_VM_LOG_DIR" + else + LOG_ARG="" + fi sudo su $STACK_USER -c "$IRONIC_SCRIPTS_DIR/create-nodes \ $IRONIC_VM_SPECS_CPU $IRONIC_VM_SPECS_RAM $IRONIC_VM_SPECS_DISK \ - amd64 $IRONIC_VM_COUNT $IRONIC_VM_NETWORK_BRIDGE $IRONIC_VM_EMULATOR" >> $IRONIC_VM_MACS_CSV_FILE + amd64 $IRONIC_VM_COUNT $IRONIC_VM_NETWORK_BRIDGE $IRONIC_VM_EMULATOR \ + $LOG_ARG" >> $IRONIC_VM_MACS_CSV_FILE } function enroll_vms { @@ -315,12 +346,19 @@ function enroll_vms { IRONIC_NET_ID=$(neutron net-list | grep private | get_field 1) local idx=0 - # work around; need to know what netns neutron uses for private network - neutron port-create private + # work around; need to know what netns neutron uses for private network. + # Without knowing how to interconnect the networks, PXE won't work properly + # for fake baremetal instances. The network should be configured prior all + # the instances operation. If we don't do this, the first port creation + # only happens in the middle of fake baremetal instance's spawning by nova, + # so we'll end up with unbootable fake baremetal VM due to broken PXE. + PORT_ID=$(neutron port-create private | grep " id " | get_field 2) while read MAC; do NODE_ID=$(ironic node-create --chassis_uuid $CHASSIS_ID --driver pxe_ssh \ + -i pxe_deploy_kernel=$IRONIC_DEPLOY_KERNEL_ID \ + -i pxe_deploy_ramdisk=$IRONIC_DEPLOY_RAMDISK_ID \ -i ssh_virt_type=$IRONIC_SSH_VIRT_TYPE \ -i ssh_address=$IRONIC_VM_SSH_ADDRESS \ -i ssh_port=$IRONIC_VM_SSH_PORT \ @@ -339,7 +377,12 @@ function enroll_vms { done < $IRONIC_VM_MACS_CSV_FILE # create the nova flavor - nova flavor-create baremetal auto $IRONIC_VM_SPECS_RAM $IRONIC_VM_SPECS_DISK $IRONIC_VM_SPECS_CPU + adjusted_disk=$(($IRONIC_VM_SPECS_DISK - $IRONIC_VM_EPHEMERAL_DISK)) + nova flavor-create --ephemeral $IRONIC_VM_EPHEMERAL_DISK baremetal auto $IRONIC_VM_SPECS_RAM $adjusted_disk $IRONIC_VM_SPECS_CPU + # TODO(lucasagomes): Remove the 'baremetal:deploy_kernel_id' + # and 'baremetal:deploy_ramdisk_id' parameters + # from the flavor after the completion of + # https://blueprints.launchpad.net/ironic/+spec/add-node-instance-info nova flavor-key baremetal set "cpu_arch"="x86_64" "baremetal:deploy_kernel_id"="$IRONIC_DEPLOY_KERNEL_ID" "baremetal:deploy_ramdisk_id"="$IRONIC_DEPLOY_RAMDISK_ID" # intentional sleep to make sure the tag has been set to port @@ -357,13 +400,22 @@ function enroll_vms { sudo ovs-vsctl -- --if-exists del-port ovs-tap1 -- add-port br-int ovs-tap1 tag=$TAG_ID sudo ovs-vsctl -- --if-exists del-port brbm-tap1 -- add-port $IRONIC_VM_NETWORK_BRIDGE brbm-tap1 + + # Remove the port needed only for workaround. For additional info read the + # comment at the beginning of this function + neutron port-delete $PORT_ID } -function configure_tftpd { - # enable tftp natting for allowing connections to SERVICE_HOST's tftp server +function configure_iptables { + # enable tftp natting for allowing connections to HOST_IP's tftp server sudo modprobe nf_conntrack_tftp sudo modprobe nf_nat_tftp + # nodes boot from TFTP and callback to the API server listening on $HOST_IP + sudo iptables -I INPUT -d $HOST_IP -p udp --dport 69 -j ACCEPT || true + sudo iptables -I INPUT -d $HOST_IP -p tcp --dport 6385 -j ACCEPT || true +} +function configure_tftpd { if is_ubuntu; then PXEBIN=/usr/lib/syslinux/pxelinux.0 elif is_fedora; then @@ -412,31 +464,10 @@ function ironic_ssh_check { fi } -function configure_ironic_sshd { - # Ensure sshd server accepts connections from localhost only - - SSH_CONFIG=/etc/ssh/sshd_config - HOST_PORT=$IRONIC_VM_SSH_ADDRESS:$IRONIC_VM_SSH_PORT - if ! sudo grep ListenAddress $SSH_CONFIG | grep $HOST_PORT; then - echo "ListenAddress $HOST_PORT" | sudo tee -a $SSH_CONFIG - fi - - SSH_SERVICE_NAME=sshd - if is_ubuntu; then - SSH_SERVICE_NAME=ssh - fi - - restart_service $SSH_SERVICE_NAME - # to ensure ssh service is up and running - sleep 3 - ironic_ssh_check $IRONIC_SSH_KEY_DIR/$IRONIC_SSH_KEY_FILENAME $IRONIC_VM_SSH_ADDRESS $IRONIC_VM_SSH_PORT $IRONIC_SSH_USERNAME 10 - -} - function configure_ironic_auxiliary { configure_ironic_dirs configure_ironic_ssh_keypair - configure_ironic_sshd + ironic_ssh_check $IRONIC_SSH_KEY_DIR/$IRONIC_SSH_KEY_FILENAME $IRONIC_VM_SSH_ADDRESS $IRONIC_VM_SSH_PORT $IRONIC_SSH_USERNAME 10 } # build deploy kernel+ramdisk, then upload them to glance @@ -496,19 +527,6 @@ function prepare_baremetal_basic_ops { fi done - SCREEN_NAME=${SCREEN_NAME:-stack} - SERVICE_DIR=${SERVICE_DIR:-${DEST}/status} - - # stop all nova services - stop_nova || true - - # remove any nova services failure status - find $SERVICE_DIR/$SCREEN_NAME -name 'n-*.failure' -exec rm -f '{}' \; - - # start them again - start_nova_api - start_nova - TOKEN=$(keystone token-get | grep ' id ' | get_field 2) die_if_not_set $LINENO TOKEN "Keystone fail to get token" @@ -520,6 +538,12 @@ function prepare_baremetal_basic_ops { create_bridge_and_vms enroll_vms configure_tftpd + configure_iptables + + # restart nova-compute to ensure its resource tracking is up to + # date with newly enrolled nodes + stop_nova_compute || true + start_nova_compute } function cleanup_baremetal_basic_ops { @@ -534,6 +558,10 @@ function cleanup_baremetal_basic_ops { sudo su $STACK_USER -c "$IRONIC_SCRIPTS_DIR/cleanup-nodes $IRONIC_VM_COUNT $IRONIC_VM_NETWORK_BRIDGE" sudo rm -rf /etc/xinetd.d/tftp /etc/init/tftpd-hpa.override restart_service xinetd + sudo iptables -D INPUT -d $HOST_IP -p udp --dport 69 -j ACCEPT || true + sudo iptables -D INPUT -d $HOST_IP -p tcp --dport 6385 -j ACCEPT || true + sudo rmmod nf_conntrack_tftp || true + sudo rmmod nf_nat_tftp || true } # Restore xtrace + pipefail diff --git a/lib/keystone b/lib/keystone index b31cc57a56..e0ac68be12 100644 --- a/lib/keystone +++ b/lib/keystone @@ -55,7 +55,7 @@ KEYSTONE_IDENTITY_BACKEND=${KEYSTONE_IDENTITY_BACKEND:-sql} KEYSTONE_ASSIGNMENT_BACKEND=${KEYSTONE_ASSIGNMENT_BACKEND:-sql} # Select Keystone's token format -# Choose from 'UUID' and 'PKI' +# Choose from 'UUID', 'PKI', or 'PKIZ' KEYSTONE_TOKEN_FORMAT=${KEYSTONE_TOKEN_FORMAT:-PKI} # Set Keystone interface configuration @@ -87,7 +87,6 @@ if is_ssl_enabled_service "key"; then KEYSTONE_SERVICE_PROTOCOL="https" fi - # Functions # --------- # cleanup_keystone() - Remove residual data files, anything left over from previous @@ -103,18 +102,20 @@ function cleanup_keystone { function _cleanup_keystone_apache_wsgi { sudo rm -f $KEYSTONE_WSGI_DIR/*.wsgi disable_apache_site keystone - sudo rm -f /etc/$APACHE_NAME/$APACHE_CONF_DIR/keystone + sudo rm -f $(apache_site_config_for keystone) } # _config_keystone_apache_wsgi() - Set WSGI config files of Keystone function _config_keystone_apache_wsgi { sudo mkdir -p $KEYSTONE_WSGI_DIR + local keystone_apache_conf=$(apache_site_config_for keystone) + # copy proxy vhost and wsgi file sudo cp $KEYSTONE_DIR/httpd/keystone.py $KEYSTONE_WSGI_DIR/main sudo cp $KEYSTONE_DIR/httpd/keystone.py $KEYSTONE_WSGI_DIR/admin - sudo cp $FILES/apache-keystone.template /etc/$APACHE_NAME/$APACHE_CONF_DIR/keystone + sudo cp $FILES/apache-keystone.template $keystone_apache_conf sudo sed -e " s|%PUBLICPORT%|$KEYSTONE_SERVICE_PORT|g; s|%ADMINPORT%|$KEYSTONE_AUTH_PORT|g; @@ -122,7 +123,7 @@ function _config_keystone_apache_wsgi { s|%PUBLICWSGI%|$KEYSTONE_WSGI_DIR/main|g; s|%ADMINWSGI%|$KEYSTONE_WSGI_DIR/admin|g; s|%USER%|$STACK_USER|g - " -i /etc/$APACHE_NAME/$APACHE_CONF_DIR/keystone + " -i $keystone_apache_conf enable_apache_site keystone } @@ -201,6 +202,8 @@ function configure_keystone { if [[ "$KEYSTONE_TOKEN_FORMAT" = "UUID" ]]; then iniset $KEYSTONE_CONF token provider keystone.token.providers.uuid.Provider + elif [[ "$KEYSTONE_TOKEN_FORMAT" = "PKIZ" ]]; then + iniset $KEYSTONE_CONF token provider keystone.token.providers.pkiz.Provider fi iniset $KEYSTONE_CONF database connection `database_connection_url keystone` @@ -253,11 +256,17 @@ function configure_keystone { fi # Format logging - if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ]; then + if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ] && ! is_apache_enabled_service key ; then setup_colorized_logging $KEYSTONE_CONF DEFAULT fi if is_apache_enabled_service key; then + iniset $KEYSTONE_CONF DEFAULT debug "True" + # Eliminate the %(asctime)s.%(msecs)03d from the log format strings + iniset $KEYSTONE_CONF DEFAULT logging_context_format_string "%(process)d %(levelname)s %(name)s [%(request_id)s %(user_identity)s] %(instance)s%(message)s" + iniset $KEYSTONE_CONF DEFAULT logging_default_format_string "%(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s" + iniset $KEYSTONE_CONF DEFAULT logging_debug_format_suffix "%(funcName)s %(pathname)s:%(lineno)d" + iniset $KEYSTONE_CONF DEFAULT logging_exception_prefix "%(process)d TRACE %(name)s %(instance)s" _config_keystone_apache_wsgi fi } @@ -375,7 +384,7 @@ function init_keystone { # Initialize keystone database $KEYSTONE_DIR/bin/keystone-manage db_sync - if [[ "$KEYSTONE_TOKEN_FORMAT" == "PKI" ]]; then + if [[ "$KEYSTONE_TOKEN_FORMAT" == "PKI" || "$KEYSTONE_TOKEN_FORMAT" == "PKIZ" ]]; then # Set up certificates rm -rf $KEYSTONE_CONF_DIR/ssl $KEYSTONE_DIR/bin/keystone-manage pki_setup @@ -450,6 +459,8 @@ function start_keystone { function stop_keystone { # Kill the Keystone screen window screen_stop key + # Cleanup the WSGI files and VHOST + _cleanup_keystone_apache_wsgi } diff --git a/lib/marconi b/lib/marconi index fd1c35132a..20bf0e1c14 100644 --- a/lib/marconi +++ b/lib/marconi @@ -42,7 +42,7 @@ MARCONI_AUTH_CACHE_DIR=${MARCONI_AUTH_CACHE_DIR:-/var/cache/marconi} MARCONI_BIN_DIR=$(get_python_exec_prefix) # Set up database backend -MARCONI_BACKEND=${MARCONI_BACKEND:-mongodb} +MARCONI_BACKEND=${MARCONI_BACKEND:-sqlite} # Set Marconi repository @@ -109,8 +109,8 @@ function configure_marconi { if [ "$MARCONI_BACKEND" = 'mysql' ] || [ "$MARCONI_BACKEND" = 'postgresql' ] ; then iniset $MARCONI_CONF drivers storage sqlalchemy iniset $MARCONI_CONF 'drivers:storage:sqlalchemy' uri `database_connection_url marconi` - else - iniset $MARCONI_CONF drivers storage mongodb + elif [ "$MARCONI_BACKEND" = 'mongodb' ] ; then + iniset $MARCONI_CONF drivers storage mongodb iniset $MARCONI_CONF 'drivers:storage:mongodb' uri mongodb://localhost:27017/marconi configure_mongodb cleanup_marconi @@ -154,7 +154,12 @@ function install_marconiclient { # start_marconi() - Start running processes, including screen function start_marconi { - screen_it marconi-server "marconi-server --config-file $MARCONI_CONF" + if [[ "$USE_SCREEN" = "False" ]]; then + screen_it marconi-server "marconi-server --config-file $MARCONI_CONF --daemon" + else + screen_it marconi-server "marconi-server --config-file $MARCONI_CONF" + fi + echo "Waiting for Marconi to start..." if ! timeout $SERVICE_TIMEOUT sh -c "while ! wget --no-proxy -q -O- $MARCONI_SERVICE_PROTOCOL://$MARCONI_SERVICE_HOST:$MARCONI_SERVICE_PORT/v1/health; do sleep 1; done"; then die $LINENO "Marconi did not start" diff --git a/lib/neutron b/lib/neutron index 294fface3d..e9182864fa 100644 --- a/lib/neutron +++ b/lib/neutron @@ -110,11 +110,14 @@ Q_USE_DEBUG_COMMAND=${Q_USE_DEBUG_COMMAND:-False} Q_ROUTER_NAME=${Q_ROUTER_NAME:-router1} # nova vif driver that all plugins should use NOVA_VIF_DRIVER=${NOVA_VIF_DRIVER:-"nova.virt.libvirt.vif.LibvirtGenericVIFDriver"} -Q_NOTIFY_NOVA_PORT_STATUS_CHANGE=${Q_NOTIFY_NOVA_PORT_STATUS_CHANGE:-True} -Q_NOTIFY_NOVA_ON_PORT_DATA_CHANGES=${Q_NOTIFY_NOVA_PORT_CHANGE:-True} +Q_NOTIFY_NOVA_PORT_STATUS_CHANGES=${Q_NOTIFY_NOVA_PORT_STATUS_CHANGES:-True} +Q_NOTIFY_NOVA_PORT_DATA_CHANGES=${Q_NOTIFY_NOVA_PORT_DATA_CHANGES:-True} VIF_PLUGGING_IS_FATAL=${VIF_PLUGGING_IS_FATAL:-True} VIF_PLUGGING_TIMEOUT=${VIF_PLUGGING_TIMEOUT:-300} +## Provider Network Information +PROVIDER_SUBNET_NAME=${PROVIDER_SUBNET_NAME:-"provider_net"} + # The next two variables are configured by plugin # e.g. _configure_neutron_l3_agent or lib/neutron_plugins/* # @@ -127,6 +130,10 @@ Q_L3_ROUTER_PER_TENANT=${Q_L3_ROUTER_PER_TENANT:-False} # See _configure_neutron_common() for details about setting it up declare -a Q_PLUGIN_EXTRA_CONF_FILES +# List of (optional) config files for VPN device drivers to use with +# the neutron-q-vpn agent +declare -a Q_VPN_EXTRA_CONF_FILES + Q_RR_CONF_FILE=$NEUTRON_CONF_DIR/rootwrap.conf if [[ "$Q_USE_ROOTWRAP" == "False" ]]; then @@ -274,7 +281,7 @@ function configure_neutron { if is_service_enabled q-fwaas; then _configure_neutron_fwaas fi - if is_service_enabled q-svc; then + if is_service_enabled q-agt q-svc; then _configure_neutron_service fi if is_service_enabled q-agt; then @@ -312,7 +319,7 @@ function create_nova_conf_neutron { # set NOVA_VIF_DRIVER and optionally set options in nova_conf neutron_plugin_create_nova_conf - iniset $NOVA_CONF DEFAULT libvirt_vif_driver "$NOVA_VIF_DRIVER" + iniset $NOVA_CONF libvirt vif_driver "$NOVA_VIF_DRIVER" iniset $NOVA_CONF DEFAULT linuxnet_interface_driver "$LINUXNET_VIF_DRIVER" if is_service_enabled q-meta; then iniset $NOVA_CONF DEFAULT service_neutron_metadata_proxy "True" @@ -391,6 +398,15 @@ function create_neutron_initial_network { die_if_not_set $LINENO SUBNET_ID "Failure creating SUBNET_ID for $TENANT_ID" sudo ifconfig $OVS_PHYSICAL_BRIDGE up sudo route add default gw $NETWORK_GATEWAY dev $OVS_PHYSICAL_BRIDGE + elif is_provider_network; then + die_if_not_set $LINENO SEGMENTATION_ID "A SEGMENTATION_ID is required to use provider networking" + die_if_not_set $LINENO PROVIDER_NETWORK_TYPE "You must specifiy the PROVIDER_NETWORK_TYPE" + NET_ID=$(neutron net-create $PHYSICAL_NETWORK --tenant_id $TENANT_ID --provider:network_type $PROVIDER_NETWORK_TYPE --provider:physical_network "$PHYSICAL_NETWORK" --provider:segmentation_id "$SEGMENTATION_ID" --router:external=true --shared | grep ' id ' | get_field 2) + SUBNET_ID=$(neutron subnet-create --tenant_id $TENANT_ID --ip_version 4 ${ALLOCATION_POOL:+--allocation-pool $ALLOCATION_POOL} --name $PROVIDER_SUBNET_NAME $NET_ID $FIXED_RANGE | grep ' id ' | get_field 2) + SUBNET_V6_ID=$(neutron subnet-create --tenant_id $TENANT_ID --ip_version 6 --ipv6-address-mode slaac --gateway $V6_NETWORK_GATEWAY --name $PROVIDER_SUBNET_NAME_V6 $NET_ID $FIXED_RANGE_V6 | grep 'id' | get_field 2) + sudo ip link set $OVS_PHYSICAL_BRIDGE up + sudo ip link set br-int up + sudo ip link set $PUBLIC_INTERFACE up else NET_ID=$(neutron net-create --tenant-id $TENANT_ID "$PRIVATE_NETWORK_NAME" | grep ' id ' | get_field 2) die_if_not_set $LINENO NET_ID "Failure creating NET_ID for $PHYSICAL_NETWORK $TENANT_ID" @@ -491,11 +507,19 @@ function start_neutron_agents { L3_CONF_FILES="--config-file $NEUTRON_CONF --config-file=$Q_L3_CONF_FILE" + if is_provider_network; then + sudo ovs-vsctl add-port $OVS_PHYSICAL_BRIDGE $PUBLIC_INTERFACE + sudo ip link set $OVS_PHYSICAL_BRIDGE up + sudo ip link set br-int up + sudo ip link set $PUBLIC_INTERFACE up + fi + if is_service_enabled q-fwaas; then L3_CONF_FILES="$L3_CONF_FILES --config-file $Q_FWAAS_CONF_FILE" + VPN_CONF_FILES="$VPN_CONF_FILES --config-file $Q_FWAAS_CONF_FILE" fi if is_service_enabled q-vpn; then - screen_it q-vpn "cd $NEUTRON_DIR && $AGENT_VPN_BINARY $L3_CONF_FILES" + screen_it q-vpn "cd $NEUTRON_DIR && $AGENT_VPN_BINARY $VPN_CONF_FILES" else screen_it q-l3 "cd $NEUTRON_DIR && python $AGENT_L3_BINARY $L3_CONF_FILES" fi @@ -658,6 +682,7 @@ function _configure_neutron_dhcp_agent { } function _configure_neutron_l3_agent { + local cfg_file Q_L3_ENABLED=True # for l3-agent, only use per tenant router if we have namespaces Q_L3_ROUTER_PER_TENANT=$Q_USE_NAMESPACE @@ -669,6 +694,15 @@ function _configure_neutron_l3_agent { Q_FWAAS_CONF_FILE=$NEUTRON_CONF_DIR/fwaas_driver.ini fi + if is_service_enabled q-vpn; then + Q_VPN_CONF_FILE=$NEUTRON_CONF_DIR/vpn_agent.ini + cp $NEUTRON_DIR/etc/vpn_agent.ini $Q_VPN_CONF_FILE + VPN_CONF_FILES="--config-file $NEUTRON_CONF --config-file=$Q_L3_CONF_FILE --config-file=$Q_VPN_CONF_FILE" + for cfg_file in ${Q_VPN_EXTRA_CONF_FILES[@]}; do + VPN_CONF_FILES+=" --config-file $cfg_file" + done + fi + cp $NEUTRON_DIR/etc/l3_agent.ini $Q_L3_CONF_FILE iniset $Q_L3_CONF_FILE DEFAULT verbose True @@ -762,10 +796,10 @@ function _configure_neutron_service { done # Configuration for neutron notifations to nova. - iniset $NEUTRON_CONF DEFAULT notify_nova_on_port_status_change $Q_NOTIFY_NOVA_PORT_STATUS_CHANGE - iniset $NEUTRON_CONF DEFAULT notify_nova_on_port_data_changes $Q_NOTIFY_NOVA_ON_PORT_DATA_CHANGES + iniset $NEUTRON_CONF DEFAULT notify_nova_on_port_status_changes $Q_NOTIFY_NOVA_PORT_STATUS_CHANGES + iniset $NEUTRON_CONF DEFAULT notify_nova_on_port_data_changes $Q_NOTIFY_NOVA_PORT_DATA_CHANGES iniset $NEUTRON_CONF DEFAULT nova_url "$NOVA_SERVICE_PROTOCOL://$NOVA_SERVICE_HOST:$NOVA_SERVICE_PORT/v2" - iniset $NEUTRON_CONF DEFAULT nova_admin_username nova $NOVA_USER + iniset $NEUTRON_CONF DEFAULT nova_admin_username nova iniset $NEUTRON_CONF DEFAULT nova_admin_password $SERVICE_PASSWORD ADMIN_TENANT_ID=$(openstack project list | awk "/ service / { print \$2 }") iniset $NEUTRON_CONF DEFAULT nova_admin_tenant_id $ADMIN_TENANT_ID @@ -984,6 +1018,13 @@ function check_neutron_third_party_integration { _neutron_third_party_do check } +function is_provider_network { + if [ "$Q_USE_PROVIDER_NETWORKING" == "True" ] && [ "$Q_L3_ENABLED" == "False" ]; then + return 0 + fi + return 1 +} + # Restore xtrace $XTRACE diff --git a/lib/neutron_plugins/bigswitch_floodlight b/lib/neutron_plugins/bigswitch_floodlight index b1b77d7606..efdd9ef794 100644 --- a/lib/neutron_plugins/bigswitch_floodlight +++ b/lib/neutron_plugins/bigswitch_floodlight @@ -2,7 +2,7 @@ # ------------------------------------ # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +BS_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/ovs_base @@ -74,4 +74,4 @@ function neutron_plugin_check_adv_test_requirements { } # Restore xtrace -$MY_XTRACE +$BS_XTRACE diff --git a/lib/neutron_plugins/brocade b/lib/neutron_plugins/brocade index 4443fa7823..e4cc754039 100644 --- a/lib/neutron_plugins/brocade +++ b/lib/neutron_plugins/brocade @@ -24,6 +24,30 @@ function neutron_plugin_configure_common { Q_PLUGIN_CLASS="neutron.plugins.brocade.NeutronPlugin.BrocadePluginV2" } +function neutron_plugin_configure_service { + + if [[ "$BROCADE_SWITCH_OS_VERSION" != "" ]]; then + iniset /$Q_PLUGIN_CONF_FILE switch osversion $BROCADE_SWITCH_OS_VERSION + fi + + if [[ "$BROCADE_SWITCH_OS_TYPE" != "" ]]; then + iniset /$Q_PLUGIN_CONF_FILE switch ostype $BROCADE_SWITCH_OS_TYPE + fi + + if [[ "$BROCADE_SWITCH_PASSWORD" != "" ]]; then + iniset /$Q_PLUGIN_CONF_FILE switch password $BROCADE_SWITCH_PASSWORD + fi + + if [[ "$BROCADE_SWITCH_USERNAME" != "" ]]; then + iniset /$Q_PLUGIN_CONF_FILE switch username $BROCADE_SWITCH_USERNAME + fi + + if [[ "$BROCADE_SWITCH_IPADDR" != "" ]]; then + iniset /$Q_PLUGIN_CONF_FILE switch address $BROCADE_SWITCH_IPADDR + fi + +} + function neutron_plugin_configure_debug_command { iniset $NEUTRON_TEST_CONFIG_FILE DEFAULT external_network_bridge } diff --git a/lib/neutron_plugins/cisco b/lib/neutron_plugins/cisco index a1b089e1a3..dccf4003c3 100644 --- a/lib/neutron_plugins/cisco +++ b/lib/neutron_plugins/cisco @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +CISCO_XTRACE=$(set +o | grep xtrace) set +o xtrace # Scecify the VSM parameters @@ -183,6 +183,15 @@ function neutron_plugin_configure_common { ovs_neutron_plugin_configure_common Q_PLUGIN_EXTRA_CONF_PATH=etc/neutron/plugins/cisco Q_PLUGIN_EXTRA_CONF_FILES=(cisco_plugins.ini) + # Copy extra config files to /etc so that they can be modified + # later according to Cisco-specific localrc settings. + mkdir -p /$Q_PLUGIN_EXTRA_CONF_PATH + local f + local extra_conf_file + for (( f=0; $f < ${#Q_PLUGIN_EXTRA_CONF_FILES[@]}; f+=1 )); do + extra_conf_file=$Q_PLUGIN_EXTRA_CONF_PATH/${Q_PLUGIN_EXTRA_CONF_FILES[$f]} + cp $NEUTRON_DIR/$extra_conf_file /$extra_conf_file + done else Q_PLUGIN_CONF_PATH=etc/neutron/plugins/cisco Q_PLUGIN_CONF_FILENAME=cisco_plugins.ini @@ -324,4 +333,4 @@ function neutron_plugin_setup_interface_driver { } # Restore xtrace -$MY_XTRACE +$CISCO_XTRACE diff --git a/lib/neutron_plugins/embrane b/lib/neutron_plugins/embrane index 62f9737e51..cce108a4a1 100644 --- a/lib/neutron_plugins/embrane +++ b/lib/neutron_plugins/embrane @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +EMBR_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/openvswitch @@ -37,4 +37,4 @@ function neutron_plugin_configure_service { } # Restore xtrace -$MY_XTRACE +$EMBR_XTRACE diff --git a/lib/neutron_plugins/ibm b/lib/neutron_plugins/ibm index 22c8578e64..3aef9d0359 100644 --- a/lib/neutron_plugins/ibm +++ b/lib/neutron_plugins/ibm @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +IBM_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/ovs_base @@ -130,4 +130,4 @@ function neutron_ovs_base_cleanup { } # Restore xtrace -$MY_XTRACE +$IBM_XTRACE diff --git a/lib/neutron_plugins/linuxbridge b/lib/neutron_plugins/linuxbridge index 362fd5b39e..113a7dfda6 100644 --- a/lib/neutron_plugins/linuxbridge +++ b/lib/neutron_plugins/linuxbridge @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +LBRIDGE_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/linuxbridge_agent @@ -15,7 +15,7 @@ function neutron_plugin_configure_common { } function neutron_plugin_configure_service { - if [[ "$ENABLE_TENANT_VLANS" = "True" ]]; then + if [[ "$ENABLE_TENANT_VLANS" == "True" ]]; then iniset /$Q_PLUGIN_CONF_FILE vlans tenant_network_type vlan else echo "WARNING - The linuxbridge plugin is using local tenant networks, with no connectivity between hosts." @@ -23,7 +23,7 @@ function neutron_plugin_configure_service { # Override ``LB_VLAN_RANGES`` and ``LB_INTERFACE_MAPPINGS`` in ``localrc`` # for more complex physical network configurations. - if [[ "$LB_VLAN_RANGES" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]]; then + if [[ "$LB_VLAN_RANGES" == "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]]; then LB_VLAN_RANGES=$PHYSICAL_NETWORK if [[ "$TENANT_VLAN_RANGE" != "" ]]; then LB_VLAN_RANGES=$LB_VLAN_RANGES:$TENANT_VLAN_RANGE @@ -53,4 +53,4 @@ function has_neutron_plugin_security_group { } # Restore xtrace -$MY_XTRACE +$LBRIDGE_XTRACE diff --git a/lib/neutron_plugins/linuxbridge_agent b/lib/neutron_plugins/linuxbridge_agent index 74799e477c..82b5fc903d 100644 --- a/lib/neutron_plugins/linuxbridge_agent +++ b/lib/neutron_plugins/linuxbridge_agent @@ -35,7 +35,7 @@ function neutron_plugin_configure_plugin_agent { # Setup physical network interface mappings. Override # ``LB_VLAN_RANGES`` and ``LB_INTERFACE_MAPPINGS`` in ``localrc`` for more # complex physical network configurations. - if [[ "$LB_INTERFACE_MAPPINGS" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]] && [[ "$LB_PHYSICAL_INTERFACE" != "" ]]; then + if [[ "$LB_INTERFACE_MAPPINGS" == "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]] && [[ "$LB_PHYSICAL_INTERFACE" != "" ]]; then LB_INTERFACE_MAPPINGS=$PHYSICAL_NETWORK:$LB_PHYSICAL_INTERFACE fi if [[ "$LB_INTERFACE_MAPPINGS" != "" ]]; then diff --git a/lib/neutron_plugins/midonet b/lib/neutron_plugins/midonet index 742e3b2f0f..c5373d656f 100644 --- a/lib/neutron_plugins/midonet +++ b/lib/neutron_plugins/midonet @@ -6,7 +6,7 @@ MIDONET_API_PORT=${MIDONET_API_PORT:-8080} MIDONET_API_URL=${MIDONET_API_URL:-http://localhost:$MIDONET_API_PORT/midonet-api} # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +MN_XTRACE=$(set +o | grep xtrace) set +o xtrace function is_neutron_ovs_base_plugin { @@ -84,4 +84,4 @@ function neutron_plugin_check_adv_test_requirements { } # Restore xtrace -$MY_XTRACE +$MN_XTRACE diff --git a/lib/neutron_plugins/ml2 b/lib/neutron_plugins/ml2 index e985dcb4a5..99663733e1 100644 --- a/lib/neutron_plugins/ml2 +++ b/lib/neutron_plugins/ml2 @@ -2,7 +2,7 @@ # ------------------------------ # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +ML2_XTRACE=$(set +o | grep xtrace) set +o xtrace # Enable this to simply and quickly enable tunneling with ML2. @@ -11,7 +11,7 @@ Q_ML2_TENANT_NETWORK_TYPE=${Q_ML2_TENANT_NETWORK_TYPE:-} # This has to be set here since the agent will set this in the config file if [[ "$Q_ML2_TENANT_NETWORK_TYPE" != "" ]]; then Q_AGENT_EXTRA_AGENT_OPTS+=(tunnel_types=$Q_ML2_TENANT_NETWORK_TYPE) -elif [[ "$ENABLE_TENANT_TUNNELS" = "True" ]]; then +elif [[ "$ENABLE_TENANT_TUNNELS" == "True" ]]; then Q_AGENT_EXTRA_AGENT_OPTS+=(tunnel_types=gre) fi @@ -60,12 +60,12 @@ function neutron_plugin_configure_common { function neutron_plugin_configure_service { if [[ "$Q_ML2_TENANT_NETWORK_TYPE" != "" ]]; then Q_SRV_EXTRA_OPTS+=(tenant_network_types=$Q_ML2_TENANT_NETWORK_TYPE) - elif [[ "$ENABLE_TENANT_TUNNELS" = "True" ]]; then + elif [[ "$ENABLE_TENANT_TUNNELS" == "True" ]]; then # This assumes you want a simple configuration, and will overwrite # Q_SRV_EXTRA_OPTS if set in addition to ENABLE_TENANT_TUNNELS. Q_SRV_EXTRA_OPTS+=(tenant_network_types=gre) Q_ML2_PLUGIN_GRE_TYPE_OPTIONS=(tunnel_id_ranges=$TENANT_TUNNEL_RANGES) - elif [[ "$ENABLE_TENANT_VLANS" = "True" ]]; then + elif [[ "$ENABLE_TENANT_VLANS" == "True" ]]; then Q_SRV_EXTRA_OPTS+=(tenant_network_types=vlan) else echo "WARNING - The ml2 plugin is using local tenant networks, with no connectivity between hosts." @@ -74,7 +74,7 @@ function neutron_plugin_configure_service { # Allow for overrding VLAN configuration (for example, to configure provider # VLANs) by first checking if Q_ML2_PLUGIN_VLAN_TYPE_OPTIONS is set. if [ "$Q_ML2_PLUGIN_VLAN_TYPE_OPTIONS" == "" ]; then - if [[ "$ML2_VLAN_RANGES" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]]; then + if [[ "$ML2_VLAN_RANGES" == "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]]; then ML2_VLAN_RANGES=$PHYSICAL_NETWORK if [[ "$TENANT_VLAN_RANGE" != "" ]]; then ML2_VLAN_RANGES=$ML2_VLAN_RANGES:$TENANT_VLAN_RANGE @@ -119,4 +119,4 @@ function has_neutron_plugin_security_group { } # Restore xtrace -$MY_XTRACE +$ML2_XTRACE diff --git a/lib/neutron_plugins/nec b/lib/neutron_plugins/nec index 6d4bfca244..d76f7d4aaf 100644 --- a/lib/neutron_plugins/nec +++ b/lib/neutron_plugins/nec @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +NEC_XTRACE=$(set +o | grep xtrace) set +o xtrace # Configuration parameters @@ -127,4 +127,4 @@ function neutron_plugin_check_adv_test_requirements { } # Restore xtrace -$MY_XTRACE +$NEC_XTRACE diff --git a/lib/neutron_plugins/nuage b/lib/neutron_plugins/nuage index 3649f39bfd..86f09d2b54 100644 --- a/lib/neutron_plugins/nuage +++ b/lib/neutron_plugins/nuage @@ -2,7 +2,7 @@ # ---------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +NU_XTRACE=$(set +o | grep xtrace) set +o xtrace function neutron_plugin_create_nova_conf { @@ -66,4 +66,4 @@ function has_neutron_plugin_security_group { } # Restore xtrace -$MY_XTRACE +$NU_XTRACE diff --git a/lib/neutron_plugins/ofagent_agent b/lib/neutron_plugins/ofagent_agent index 724df41d4c..b8321f3375 100644 --- a/lib/neutron_plugins/ofagent_agent +++ b/lib/neutron_plugins/ofagent_agent @@ -2,7 +2,7 @@ # ---------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +OFA_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/ovs_base @@ -46,7 +46,7 @@ function neutron_plugin_configure_plugin_agent { fi # Enable tunnel networks if selected - if [[ "$OVS_ENABLE_TUNNELING" = "True" ]]; then + if [[ "$OVS_ENABLE_TUNNELING" == "True" ]]; then # Verify tunnels are supported # REVISIT - also check kernel module support for GRE and patch ports OVS_VERSION=`ovs-vsctl --version | head -n 1 | grep -E -o "[0-9]+\.[0-9]+"` @@ -60,7 +60,7 @@ function neutron_plugin_configure_plugin_agent { # Setup physical network bridge mappings. Override # ``OVS_VLAN_RANGES`` and ``OVS_BRIDGE_MAPPINGS`` in ``localrc`` for more # complex physical network configurations. - if [[ "$OVS_BRIDGE_MAPPINGS" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]] && [[ "$OVS_PHYSICAL_BRIDGE" != "" ]]; then + if [[ "$OVS_BRIDGE_MAPPINGS" == "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]] && [[ "$OVS_PHYSICAL_BRIDGE" != "" ]]; then OVS_BRIDGE_MAPPINGS=$PHYSICAL_NETWORK:$OVS_PHYSICAL_BRIDGE # Configure bridge manually with physical interface as port for multi-node @@ -91,4 +91,4 @@ function neutron_plugin_check_adv_test_requirements { } # Restore xtrace -$MY_XTRACE +$OFA_XTRACE diff --git a/lib/neutron_plugins/oneconvergence b/lib/neutron_plugins/oneconvergence index 0aebff629c..06f1eee8c7 100644 --- a/lib/neutron_plugins/oneconvergence +++ b/lib/neutron_plugins/oneconvergence @@ -1,7 +1,7 @@ # Neutron One Convergence plugin # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +OC_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/ovs_base @@ -73,4 +73,4 @@ function neutron_plugin_create_nova_conf { } # Restore xtrace -$MY_XTRACE +$OC_XTRACE diff --git a/lib/neutron_plugins/openvswitch b/lib/neutron_plugins/openvswitch index bdbc5a9367..fc81092682 100644 --- a/lib/neutron_plugins/openvswitch +++ b/lib/neutron_plugins/openvswitch @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +OVS_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/openvswitch_agent @@ -15,10 +15,10 @@ function neutron_plugin_configure_common { } function neutron_plugin_configure_service { - if [[ "$ENABLE_TENANT_TUNNELS" = "True" ]]; then + if [[ "$ENABLE_TENANT_TUNNELS" == "True" ]]; then iniset /$Q_PLUGIN_CONF_FILE ovs tenant_network_type gre iniset /$Q_PLUGIN_CONF_FILE ovs tunnel_id_ranges $TENANT_TUNNEL_RANGES - elif [[ "$ENABLE_TENANT_VLANS" = "True" ]]; then + elif [[ "$ENABLE_TENANT_VLANS" == "True" ]]; then iniset /$Q_PLUGIN_CONF_FILE ovs tenant_network_type vlan else echo "WARNING - The openvswitch plugin is using local tenant networks, with no connectivity between hosts." @@ -26,7 +26,7 @@ function neutron_plugin_configure_service { # Override ``OVS_VLAN_RANGES`` and ``OVS_BRIDGE_MAPPINGS`` in ``localrc`` # for more complex physical network configurations. - if [[ "$OVS_VLAN_RANGES" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]]; then + if [[ "$OVS_VLAN_RANGES" == "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]]; then OVS_VLAN_RANGES=$PHYSICAL_NETWORK if [[ "$TENANT_VLAN_RANGE" != "" ]]; then OVS_VLAN_RANGES=$OVS_VLAN_RANGES:$TENANT_VLAN_RANGE @@ -37,7 +37,7 @@ function neutron_plugin_configure_service { fi # Enable tunnel networks if selected - if [[ $OVS_ENABLE_TUNNELING = "True" ]]; then + if [[ $OVS_ENABLE_TUNNELING == "True" ]]; then iniset /$Q_PLUGIN_CONF_FILE ovs enable_tunneling True fi @@ -57,4 +57,4 @@ function has_neutron_plugin_security_group { } # Restore xtrace -$MY_XTRACE +$OVS_XTRACE diff --git a/lib/neutron_plugins/openvswitch_agent b/lib/neutron_plugins/openvswitch_agent index 3a2bdc316a..fbc013f565 100644 --- a/lib/neutron_plugins/openvswitch_agent +++ b/lib/neutron_plugins/openvswitch_agent @@ -2,16 +2,16 @@ # ----------------------------- # Save trace setting -PLUGIN_XTRACE=$(set +o | grep xtrace) +OVSA_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/ovs_base function neutron_plugin_create_nova_conf { _neutron_ovs_base_configure_nova_vif_driver - if [ "$VIRT_DRIVER" = 'xenserver' ]; then - iniset $NOVA_CONF DEFAULT xenapi_vif_driver nova.virt.xenapi.vif.XenAPIOpenVswitchDriver - iniset $NOVA_CONF DEFAULT xenapi_ovs_integration_bridge $XEN_INTEGRATION_BRIDGE + if [ "$VIRT_DRIVER" == 'xenserver' ]; then + iniset $NOVA_CONF xenserver vif_driver nova.virt.xenapi.vif.XenAPIOpenVswitchDriver + iniset $NOVA_CONF xenserver ovs_integration_bridge $XEN_INTEGRATION_BRIDGE # Disable nova's firewall so that it does not conflict with neutron iniset $NOVA_CONF DEFAULT firewall_driver nova.virt.firewall.NoopFirewallDriver fi @@ -40,7 +40,7 @@ function neutron_plugin_configure_plugin_agent { _neutron_ovs_base_configure_firewall_driver # Setup agent for tunneling - if [[ "$OVS_ENABLE_TUNNELING" = "True" ]]; then + if [[ "$OVS_ENABLE_TUNNELING" == "True" ]]; then # Verify tunnels are supported # REVISIT - also check kernel module support for GRE and patch ports OVS_VERSION=`ovs-vsctl --version | head -n 1 | grep -E -o "[0-9]+\.[0-9]+"` @@ -54,7 +54,7 @@ function neutron_plugin_configure_plugin_agent { # Setup physical network bridge mappings. Override # ``OVS_VLAN_RANGES`` and ``OVS_BRIDGE_MAPPINGS`` in ``localrc`` for more # complex physical network configurations. - if [[ "$OVS_BRIDGE_MAPPINGS" = "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]] && [[ "$OVS_PHYSICAL_BRIDGE" != "" ]]; then + if [[ "$OVS_BRIDGE_MAPPINGS" == "" ]] && [[ "$PHYSICAL_NETWORK" != "" ]] && [[ "$OVS_PHYSICAL_BRIDGE" != "" ]]; then OVS_BRIDGE_MAPPINGS=$PHYSICAL_NETWORK:$OVS_PHYSICAL_BRIDGE # Configure bridge manually with physical interface as port for multi-node @@ -65,7 +65,7 @@ function neutron_plugin_configure_plugin_agent { fi AGENT_BINARY="$NEUTRON_BIN_DIR/neutron-openvswitch-agent" - if [ "$VIRT_DRIVER" = 'xenserver' ]; then + if [ "$VIRT_DRIVER" == 'xenserver' ]; then # Make a copy of our config for domU sudo cp /$Q_PLUGIN_CONF_FILE "/$Q_PLUGIN_CONF_FILE.domu" @@ -128,4 +128,4 @@ function neutron_plugin_check_adv_test_requirements { } # Restore xtrace -$PLUGIN_XTRACE +$OVSA_XTRACE diff --git a/lib/neutron_plugins/ovs_base b/lib/neutron_plugins/ovs_base index 0a2ba58fbb..1e293a187e 100644 --- a/lib/neutron_plugins/ovs_base +++ b/lib/neutron_plugins/ovs_base @@ -2,7 +2,7 @@ # ------------------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +OVSB_XTRACE=$(set +o | grep xtrace) set +o xtrace OVS_BRIDGE=${OVS_BRIDGE:-br-int} @@ -44,9 +44,8 @@ function _neutron_ovs_base_install_agent_packages { # Ensure that the service is started restart_service openvswitch elif is_suse; then - install_package openvswitch + install_package openvswitch-switch restart_service openvswitch-switch - restart_service openvswitch-controller fi } @@ -77,4 +76,4 @@ function _neutron_ovs_base_configure_nova_vif_driver { } # Restore xtrace -$MY_XTRACE +$OVSB_XTRACE diff --git a/lib/neutron_plugins/plumgrid b/lib/neutron_plugins/plumgrid index 19f94cb78c..178bca7dc2 100644 --- a/lib/neutron_plugins/plumgrid +++ b/lib/neutron_plugins/plumgrid @@ -3,7 +3,7 @@ # ------------------------------------ # Save trace settings -MY_XTRACE=$(set +o | grep xtrace) +PG_XTRACE=$(set +o | grep xtrace) set +o xtrace function neutron_plugin_create_nova_conf { @@ -24,14 +24,16 @@ function neutron_plugin_configure_common { PLUMGRID_ADMIN=${PLUMGRID_ADMIN:-username} PLUMGRID_PASSWORD=${PLUMGRID_PASSWORD:-password} PLUMGRID_TIMEOUT=${PLUMGRID_TIMEOUT:-70} + PLUMGRID_DRIVER=${PLUMGRID_DRIVER:-neutron.plugins.plumgrid.drivers.fake_plumlib.Plumlib} } function neutron_plugin_configure_service { - iniset /$Q_PLUGIN_CONF_FILE PLUMgridDirector director_server $PLUMGRID_DIRECTOR_IP - iniset /$Q_PLUGIN_CONF_FILE PLUMgridDirector director_server_port $PLUMGRID_DIRECTOR_PORT - iniset /$Q_PLUGIN_CONF_FILE PLUMgridDirector username $PLUMGRID_ADMIN - iniset /$Q_PLUGIN_CONF_FILE PLUMgridDirector password $PLUMGRID_PASSWORD - iniset /$Q_PLUGIN_CONF_FILE PLUMgridDirector servertimeout $PLUMGRID_TIMEOUT + iniset /$Q_PLUGIN_CONF_FILE plumgriddirector director_server $PLUMGRID_DIRECTOR_IP + iniset /$Q_PLUGIN_CONF_FILE plumgriddirector director_server_port $PLUMGRID_DIRECTOR_PORT + iniset /$Q_PLUGIN_CONF_FILE plumgriddirector username $PLUMGRID_ADMIN + iniset /$Q_PLUGIN_CONF_FILE plumgriddirector password $PLUMGRID_PASSWORD + iniset /$Q_PLUGIN_CONF_FILE plumgriddirector servertimeout $PLUMGRID_TIMEOUT + iniset /$Q_PLUGIN_CONF_FILE plumgriddirector driver $PLUMGRID_DRIVER } function neutron_plugin_configure_debug_command { @@ -52,4 +54,4 @@ function neutron_plugin_check_adv_test_requirements { is_service_enabled q-agt && is_service_enabled q-dhcp && return 0 } # Restore xtrace -$MY_XTRACE +$PG_XTRACE diff --git a/lib/neutron_plugins/ryu b/lib/neutron_plugins/ryu index 9ae36d38fa..ceb89faf6e 100644 --- a/lib/neutron_plugins/ryu +++ b/lib/neutron_plugins/ryu @@ -2,7 +2,7 @@ # ------------------ # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +RYU_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/ovs_base @@ -77,4 +77,4 @@ function neutron_plugin_check_adv_test_requirements { } # Restore xtrace -$MY_XTRACE +$RYU_XTRACE diff --git a/lib/neutron_plugins/services/firewall b/lib/neutron_plugins/services/firewall index ab6c32426a..b5253dbeef 100644 --- a/lib/neutron_plugins/services/firewall +++ b/lib/neutron_plugins/services/firewall @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +FW_XTRACE=$(set +o | grep xtrace) set +o xtrace FWAAS_PLUGIN=neutron.services.firewall.fwaas_plugin.FirewallPlugin @@ -24,4 +24,4 @@ function neutron_fwaas_stop { } # Restore xtrace -$MY_XTRACE +$FW_XTRACE diff --git a/lib/neutron_plugins/services/loadbalancer b/lib/neutron_plugins/services/loadbalancer index 531f52f9f5..78e7738345 100644 --- a/lib/neutron_plugins/services/loadbalancer +++ b/lib/neutron_plugins/services/loadbalancer @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +LB_XTRACE=$(set +o | grep xtrace) set +o xtrace @@ -48,4 +48,4 @@ function neutron_lbaas_stop { } # Restore xtrace -$MY_XTRACE +$LB_XTRACE diff --git a/lib/neutron_plugins/services/metering b/lib/neutron_plugins/services/metering index 0e5f75b27b..51123e2ff8 100644 --- a/lib/neutron_plugins/services/metering +++ b/lib/neutron_plugins/services/metering @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +METER_XTRACE=$(set +o | grep xtrace) set +o xtrace @@ -27,4 +27,4 @@ function neutron_metering_stop { } # Restore xtrace -$MY_XTRACE +$METER_XTRACE diff --git a/lib/neutron_plugins/services/vpn b/lib/neutron_plugins/services/vpn index e56d3613c2..d920ba621f 100644 --- a/lib/neutron_plugins/services/vpn +++ b/lib/neutron_plugins/services/vpn @@ -2,7 +2,7 @@ # --------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +VPN_XTRACE=$(set +o | grep xtrace) set +o xtrace @@ -30,4 +30,4 @@ function neutron_vpn_stop { } # Restore xtrace -$MY_XTRACE +$VPN_XTRACE diff --git a/lib/neutron_plugins/vmware_nsx b/lib/neutron_plugins/vmware_nsx index fe793549f7..c7672dbb4a 100644 --- a/lib/neutron_plugins/vmware_nsx +++ b/lib/neutron_plugins/vmware_nsx @@ -2,7 +2,7 @@ # ------------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +NSX_XTRACE=$(set +o | grep xtrace) set +o xtrace source $TOP_DIR/lib/neutron_plugins/ovs_base @@ -58,7 +58,7 @@ function neutron_plugin_configure_dhcp_agent { function neutron_plugin_configure_l3_agent { # VMware NSX plugin does not run L3 agent - die $LINENO "q-l3 should must not be executed with VMware NSX plugin!" + die $LINENO "q-l3 should not be executed with VMware NSX plugin!" } function neutron_plugin_configure_plugin_agent { @@ -146,4 +146,4 @@ function neutron_plugin_check_adv_test_requirements { } # Restore xtrace -$MY_XTRACE +$NSX_XTRACE diff --git a/lib/neutron_thirdparty/bigswitch_floodlight b/lib/neutron_thirdparty/bigswitch_floodlight index f03de56295..033731e27c 100644 --- a/lib/neutron_thirdparty/bigswitch_floodlight +++ b/lib/neutron_thirdparty/bigswitch_floodlight @@ -2,7 +2,7 @@ # ------------------------------------------ # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +BS3_XTRACE=$(set +o | grep xtrace) set +o xtrace BS_FL_CONTROLLERS_PORT=${BS_FL_CONTROLLERS_PORT:-localhost:80} @@ -49,4 +49,4 @@ function check_bigswitch_floodlight { } # Restore xtrace -$MY_XTRACE +$BS3_XTRACE diff --git a/lib/neutron_thirdparty/midonet b/lib/neutron_thirdparty/midonet index ad417bbc29..099a66eb2d 100644 --- a/lib/neutron_thirdparty/midonet +++ b/lib/neutron_thirdparty/midonet @@ -17,7 +17,7 @@ MIDONET_CLIENT_BRANCH=${MIDONET_CLIENT_BRANCH:-master} MIDONET_CLIENT_DIR=${MIDONET_CLIENT_DIR:-$MIDONET_DIR/python-midonetclient} # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +MN3_XTRACE=$(set +o | grep xtrace) set +o xtrace function configure_midonet { @@ -46,4 +46,4 @@ function check_midonet { } # Restore xtrace -$MY_XTRACE +$MN3_XTRACE diff --git a/lib/neutron_thirdparty/ryu b/lib/neutron_thirdparty/ryu index b2c1b613fe..bbe227eeb9 100644 --- a/lib/neutron_thirdparty/ryu +++ b/lib/neutron_thirdparty/ryu @@ -2,7 +2,7 @@ # ----------------------- # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +RYU3_XTRACE=$(set +o | grep xtrace) set +o xtrace @@ -75,4 +75,4 @@ function check_ryu { } # Restore xtrace -$MY_XTRACE +$RYU3_XTRACE diff --git a/lib/neutron_thirdparty/trema b/lib/neutron_thirdparty/trema index d465ac753e..3e59ef2b28 100644 --- a/lib/neutron_thirdparty/trema +++ b/lib/neutron_thirdparty/trema @@ -13,7 +13,7 @@ TREMA_APPS_REPO=${TREMA_APPS_REPO:-https://github.com/trema/apps.git} TREMA_APPS_BRANCH=${TREMA_APPS_BRANCH:-master} # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +TREMA3_XTRACE=$(set +o | grep xtrace) set +o xtrace TREMA_DIR=${TREMA_DIR:-$DEST/trema} @@ -28,7 +28,7 @@ TREMA_TMP_DIR=$TREMA_DATA_DIR/trema TREMA_LOG_LEVEL=${TREMA_LOG_LEVEL:-info} TREMA_SS_CONFIG=$TREMA_SS_ETC_DIR/sliceable.conf -TREMA_SS_APACHE_CONFIG=/etc/apache2/sites-available/sliceable_switch.conf +TREMA_SS_APACHE_CONFIG=$(apache_site_config_for sliceable_switch) # configure_trema - Set config files, create data dirs, etc function configure_trema { @@ -61,8 +61,9 @@ function init_trema { sudo cp $TREMA_SS_DIR/apache/sliceable_switch $TREMA_SS_APACHE_CONFIG sudo sed -i -e "s|/home/sliceable_switch/script|$TREMA_SS_SCRIPT_DIR|" \ $TREMA_SS_APACHE_CONFIG + # TODO(gabriel-bezerra): use some function from lib/apache to enable these modules sudo a2enmod rewrite actions - sudo a2ensite sliceable_switch.conf + enable_apache_site sliceable_switch cp $TREMA_SS_DIR/sliceable_switch_null.conf $TREMA_SS_CONFIG sed -i -e "s|^\$apps_dir.*$|\$apps_dir = \"$TREMA_DIR/apps\"|" \ @@ -98,8 +99,7 @@ function install_trema { } function start_trema { - # APACHE_NAME is defined in init_horizon (in lib/horizon) - restart_service $APACHE_NAME + restart_apache_server sudo LOGGING_LEVEL=$TREMA_LOG_LEVEL TREMA_TMP=$TREMA_TMP_DIR \ trema run -d -c $TREMA_SS_CONFIG @@ -114,4 +114,4 @@ function check_trema { } # Restore xtrace -$MY_XTRACE +$TREMA3_XTRACE diff --git a/lib/neutron_thirdparty/vmware_nsx b/lib/neutron_thirdparty/vmware_nsx index 3fecc62560..7a76570775 100644 --- a/lib/neutron_thirdparty/vmware_nsx +++ b/lib/neutron_thirdparty/vmware_nsx @@ -11,7 +11,7 @@ # * NSX_GATEWAY_NETWORK_CIDR --> CIDR to configure br-ex, e.g. 172.24.4.211/24 # Save trace setting -MY_XTRACE=$(set +o | grep xtrace) +NSX3_XTRACE=$(set +o | grep xtrace) set +o xtrace # This is the interface that connects the Devstack instance @@ -83,4 +83,4 @@ function check_vmware_nsx { } # Restore xtrace -$MY_XTRACE +$NSX3_XTRACE diff --git a/lib/nova b/lib/nova index 5cc94ec676..76929b1a46 100644 --- a/lib/nova +++ b/lib/nova @@ -75,8 +75,18 @@ QEMU_CONF=/etc/libvirt/qemu.conf # Set default defaults here as some hypervisor drivers override these PUBLIC_INTERFACE_DEFAULT=br100 -GUEST_INTERFACE_DEFAULT=eth0 FLAT_NETWORK_BRIDGE_DEFAULT=br100 +# set the GUEST_INTERFACE_DEFAULT to some interface on the box so that +# the default isn't completely crazy. This will match eth*, em*, or +# the new p* interfaces, then basically picks the first +# alphabetically. It's probably wrong, however it's less wrong than +# always using 'eth0' which doesn't exist on new Linux distros at all. +GUEST_INTERFACE_DEFAULT=$(ip link \ + | grep 'state UP' \ + | awk '{print $2}' \ + | sed 's/://' \ + | grep ^[ep] \ + | head -1) # Get hypervisor configuration # ---------------------------- @@ -236,16 +246,6 @@ function configure_nova { # Get the sample configuration file in place cp $NOVA_DIR/etc/nova/api-paste.ini $NOVA_CONF_DIR - - # Comment out the keystone configs in Nova's api-paste.ini. - # We are using nova.conf to configure this instead. - inicomment $NOVA_API_PASTE_INI filter:authtoken auth_host - inicomment $NOVA_API_PASTE_INI filter:authtoken auth_protocol - inicomment $NOVA_API_PASTE_INI filter:authtoken admin_tenant_name - inicomment $NOVA_API_PASTE_INI filter:authtoken cafile - inicomment $NOVA_API_PASTE_INI filter:authtoken admin_user - inicomment $NOVA_API_PASTE_INI filter:authtoken admin_password - inicomment $NOVA_API_PASTE_INI filter:authtoken signing_dir fi if is_service_enabled n-cpu; then @@ -303,6 +303,10 @@ function configure_nova { sudo chown -R $STACK_USER $NOVA_INSTANCES_PATH fi fi + if is_suse; then + # iscsid is not started by default + start_service iscsid + fi fi # Rebuild the config file from scratch @@ -428,18 +432,13 @@ function create_nova_conf { iniset $NOVA_CONF DEFAULT default_floating_pool "$PUBLIC_NETWORK_NAME" iniset $NOVA_CONF DEFAULT s3_host "$SERVICE_HOST" iniset $NOVA_CONF DEFAULT s3_port "$S3_SERVICE_PORT" - iniset $NOVA_CONF DEFAULT osapi_compute_extension "nova.api.openstack.compute.contrib.standard_extensions" iniset $NOVA_CONF DEFAULT my_ip "$HOST_IP" - iniset $NOVA_CONF DEFAULT osapi_compute_workers "4" - iniset $NOVA_CONF DEFAULT ec2_workers "4" - iniset $NOVA_CONF DEFAULT metadata_workers "4" - iniset $NOVA_CONF conductor workers "4" iniset $NOVA_CONF DEFAULT sql_connection `database_connection_url nova` iniset $NOVA_CONF DEFAULT instance_name_template "${INSTANCE_NAME_PREFIX}%08x" iniset $NOVA_CONF osapi_v3 enabled "True" - if is_fedora; then - # nova defaults to /usr/local/bin, but fedora pip likes to + if is_fedora || is_suse; then + # nova defaults to /usr/local/bin, but fedora and suse pip like to # install things in /usr/bin iniset $NOVA_CONF DEFAULT bindir "/usr/bin" fi @@ -468,9 +467,6 @@ function create_nova_conf { iniset $NOVA_CONF keystone_authtoken signing_dir $NOVA_AUTH_CACHE_DIR - if is_service_enabled cinder; then - iniset $NOVA_CONF DEFAULT volume_api_class "nova.volume.cinder.API" - fi if [ -n "$NOVA_STATE_PATH" ]; then iniset $NOVA_CONF DEFAULT state_path "$NOVA_STATE_PATH" iniset $NOVA_CONF DEFAULT lock_path "$NOVA_STATE_PATH" diff --git a/lib/nova_plugins/functions-libvirt b/lib/nova_plugins/functions-libvirt index 1f2b23992e..18bdf89025 100644 --- a/lib/nova_plugins/functions-libvirt +++ b/lib/nova_plugins/functions-libvirt @@ -28,6 +28,18 @@ function install_libvirt { install_package libvirt-python install_package python-libguestfs fi + + # Restart firewalld after install of libvirt to avoid a problem + # with polkit, which libvirtd brings in. See + # https://bugzilla.redhat.com/show_bug.cgi?id=1099031 + + # Note there is a difference between F20 rackspace cloud images + # and HP images used in the gate; rackspace has firewalld but hp + # cloud doesn't. RHEL6 doesn't have firewalld either. So we + # don't care if it fails. + if is_fedora; then + sudo service firewalld restart || true + fi } # Configures the installed libvirt system so that is accessible by diff --git a/lib/nova_plugins/hypervisor-baremetal b/lib/nova_plugins/hypervisor-baremetal index 2da1097027..1d4d4144df 100644 --- a/lib/nova_plugins/hypervisor-baremetal +++ b/lib/nova_plugins/hypervisor-baremetal @@ -49,7 +49,7 @@ function configure_nova_hypervisor { iniset $NOVA_CONF DEFAULT scheduler_host_manager nova.scheduler.baremetal_host_manager.BaremetalHostManager iniset $NOVA_CONF DEFAULT ram_allocation_ratio 1.0 iniset $NOVA_CONF DEFAULT reserved_host_memory_mb 0 - iniset $NOVA_CONF baremetal instance_type_extra_specs cpu_arch:$BM_CPU_ARCH + iniset $NOVA_CONF baremetal flavor_extra_specs cpu_arch:$BM_CPU_ARCH iniset $NOVA_CONF baremetal driver $BM_DRIVER iniset $NOVA_CONF baremetal power_manager $BM_POWER_MANAGER iniset $NOVA_CONF baremetal tftp_root /tftpboot diff --git a/lib/nova_plugins/hypervisor-libvirt b/lib/nova_plugins/hypervisor-libvirt index 053df3cdf5..259bf15a27 100644 --- a/lib/nova_plugins/hypervisor-libvirt +++ b/lib/nova_plugins/hypervisor-libvirt @@ -39,9 +39,9 @@ function cleanup_nova_hypervisor { # configure_nova_hypervisor - Set config files, create data dirs, etc function configure_nova_hypervisor { configure_libvirt - iniset $NOVA_CONF DEFAULT libvirt_type "$LIBVIRT_TYPE" - iniset $NOVA_CONF DEFAULT libvirt_cpu_mode "none" - iniset $NOVA_CONF DEFAULT use_usb_tablet "False" + iniset $NOVA_CONF libvirt virt_type "$LIBVIRT_TYPE" + iniset $NOVA_CONF libvirt cpu_mode "none" + iniset $NOVA_CONF libvirt use_usb_tablet "False" iniset $NOVA_CONF DEFAULT default_ephemeral_format "ext4" iniset $NOVA_CONF DEFAULT compute_driver "libvirt.LibvirtDriver" LIBVIRT_FIREWALL_DRIVER=${LIBVIRT_FIREWALL_DRIVER:-"nova.virt.libvirt.firewall.IptablesFirewallDriver"} diff --git a/lib/nova_plugins/hypervisor-xenserver b/lib/nova_plugins/hypervisor-xenserver index 10bda2cf5b..0dba471394 100644 --- a/lib/nova_plugins/hypervisor-xenserver +++ b/lib/nova_plugins/hypervisor-xenserver @@ -49,9 +49,9 @@ function configure_nova_hypervisor { fi read_password XENAPI_PASSWORD "ENTER A PASSWORD TO USE FOR XEN." iniset $NOVA_CONF DEFAULT compute_driver "xenapi.XenAPIDriver" - iniset $NOVA_CONF DEFAULT xenapi_connection_url "$XENAPI_CONNECTION_URL" - iniset $NOVA_CONF DEFAULT xenapi_connection_username "$XENAPI_USER" - iniset $NOVA_CONF DEFAULT xenapi_connection_password "$XENAPI_PASSWORD" + iniset $NOVA_CONF xenserver connection_url "$XENAPI_CONNECTION_URL" + iniset $NOVA_CONF xenserver connection_username "$XENAPI_USER" + iniset $NOVA_CONF xenserver connection_password "$XENAPI_PASSWORD" iniset $NOVA_CONF DEFAULT flat_injected "False" # Need to avoid crash due to new firewall support XEN_FIREWALL_DRIVER=${XEN_FIREWALL_DRIVER:-"nova.virt.firewall.IptablesFirewallDriver"} @@ -63,9 +63,13 @@ function configure_nova_hypervisor { local ssh_dom0 ssh_dom0="sudo -u $DOMZERO_USER ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$dom0_ip" + # Find where the plugins should go in dom0 + xen_functions=`cat $TOP_DIR/tools/xen/functions` + PLUGIN_DIR=`$ssh_dom0 "$xen_functions; set -eux; xapi_plugin_location"` + # install nova plugins to dom0 tar -czf - -C $NOVA_DIR/plugins/xenserver/xenapi/etc/xapi.d/plugins/ ./ | - $ssh_dom0 'tar -xzf - -C /etc/xapi.d/plugins/ && chmod a+x /etc/xapi.d/plugins/*' + $ssh_dom0 "tar -xzf - -C $PLUGIN_DIR && chmod a+x $PLUGIN_DIR/*" # install console logrotate script tar -czf - -C $NOVA_DIR/tools/xenserver/ rotate_xen_guest_logs.sh | diff --git a/lib/opendaylight b/lib/opendaylight index 1022e2c061..0ac7f2b0e5 100644 --- a/lib/opendaylight +++ b/lib/opendaylight @@ -35,6 +35,15 @@ source $TOP_DIR/lib/neutron_plugins/ovs_base # ODL_MGR_IP= ODL_MGR_IP=${ODL_MGR_IP:-$SERVICE_HOST} +# The ODL endpoint URL +ODL_ENDPOINT=${ODL_ENDPOINT:-http://${ODL_MGR_IP}:8080/controller/nb/v2/neutron} + +# The ODL username +ODL_USERNAME=${ODL_USERNAME:-admin} + +# The ODL password +ODL_PASSWORD=${ODL_PASSWORD:-admin} + # ODL_DIR=$DEST/opendaylight @@ -52,6 +61,9 @@ ODL_ARGS=${ODL_ARGS:-"-XX:MaxPermSize=384m"} # How long to pause after ODL starts to let it complete booting ODL_BOOT_WAIT=${ODL_BOOT_WAIT:-60} +# The physical provider network to device mapping +ODL_PROVIDER_MAPPINGS=${ODL_PROVIDER_MAPPINGS:-physnet1:eth1} + # Set up default directories @@ -80,6 +92,12 @@ function configure_opendaylight { echo "ovsdb.of.version=1.3" >> $ODL_DIR/opendaylight/configuration/config.ini } +function configure_ml2_odl { + populate_ml2_config /$Q_PLUGIN_CONF_FILE ml2_odl url=$ODL_ENDPOINT + populate_ml2_config /$Q_PLUGIN_CONF_FILE ml2_odl username=$ODL_USERNAME + populate_ml2_config /$Q_PLUGIN_CONF_FILE ml2_odl password=$ODL_PASSWORD +} + # init_opendaylight() - Initialize databases, etc. function init_opendaylight { # clean up from previous (possibly aborted) runs @@ -117,9 +135,8 @@ function install_opendaylight-compute { # Ensure that the service is started restart_service openvswitch elif is_suse; then - install_package openvswitch + install_package openvswitch-switch restart_service openvswitch-switch - restart_service openvswitch-controller fi } diff --git a/lib/oslo b/lib/oslo index 8ef179c047..dc6b550b6c 100644 --- a/lib/oslo +++ b/lib/oslo @@ -22,6 +22,7 @@ set +o xtrace # -------- CLIFF_DIR=$DEST/cliff OSLOCFG_DIR=$DEST/oslo.config +OSLODB_DIR=$DEST/oslo.db OSLOMSG_DIR=$DEST/oslo.messaging OSLORWRAP_DIR=$DEST/oslo.rootwrap OSLOVMWARE_DIR=$DEST/oslo.vmware @@ -39,28 +40,28 @@ function install_oslo { cleanup_oslo git_clone $CLIFF_REPO $CLIFF_DIR $CLIFF_BRANCH - setup_develop $CLIFF_DIR + setup_install $CLIFF_DIR git_clone $OSLOCFG_REPO $OSLOCFG_DIR $OSLOCFG_BRANCH - setup_develop $OSLOCFG_DIR + setup_install $OSLOCFG_DIR git_clone $OSLOMSG_REPO $OSLOMSG_DIR $OSLOMSG_BRANCH - setup_develop $OSLOMSG_DIR + setup_install $OSLOMSG_DIR git_clone $OSLORWRAP_REPO $OSLORWRAP_DIR $OSLORWRAP_BRANCH - setup_develop $OSLORWRAP_DIR + setup_install $OSLORWRAP_DIR git_clone $OSLOVMWARE_REPO $OSLOVMWARE_DIR $OSLOVMWARE_BRANCH - setup_develop $OSLOVMWARE_DIR + setup_install $OSLOVMWARE_DIR git_clone $PYCADF_REPO $PYCADF_DIR $PYCADF_BRANCH - setup_develop $PYCADF_DIR + setup_install $PYCADF_DIR git_clone $STEVEDORE_REPO $STEVEDORE_DIR $STEVEDORE_BRANCH - setup_develop $STEVEDORE_DIR + setup_install $STEVEDORE_DIR git_clone $TASKFLOW_REPO $TASKFLOW_DIR $TASKFLOW_BRANCH - setup_develop $TASKFLOW_DIR + setup_install $TASKFLOW_DIR } # cleanup_oslo() - purge possibly old versions of oslo diff --git a/lib/sahara b/lib/sahara index 1ff0cf9c26..d56cf1b444 100644 --- a/lib/sahara +++ b/lib/sahara @@ -35,6 +35,8 @@ SAHARA_SERVICE_PROTOCOL=${SAHARA_SERVICE_PROTOCOL:-$SERVICE_PROTOCOL} SAHARA_AUTH_CACHE_DIR=${SAHARA_AUTH_CACHE_DIR:-/var/cache/sahara} +SAHARA_ENABLED_PLUGINS=${SAHARA_ENABLED_PLUGINS:-vanilla,hdp,fake} + # Support entry points installation of console scripts if [[ -d $SAHARA_DIR/bin ]]; then SAHARA_BIN_DIR=$SAHARA_DIR/bin @@ -109,15 +111,18 @@ function configure_sahara { rm -rf $SAHARA_AUTH_CACHE_DIR/* # Set obsolete keystone auth configs for backward compatibility - iniset $SAHARA_CONF_FILE DEFAULT os_auth_host $KEYSTONE_SERVICE_HOST - iniset $SAHARA_CONF_FILE DEFAULT os_auth_port $KEYSTONE_SERVICE_PORT - iniset $SAHARA_CONF_FILE DEFAULT os_auth_protocol $KEYSTONE_SERVICE_PROTOCOL + iniset $SAHARA_CONF_FILE DEFAULT os_auth_host $KEYSTONE_AUTH_HOST + iniset $SAHARA_CONF_FILE DEFAULT os_auth_port $KEYSTONE_AUTH_PORT + iniset $SAHARA_CONF_FILE DEFAULT os_auth_protocol $KEYSTONE_AUTH_PROTOCOL iniset $SAHARA_CONF_FILE DEFAULT os_admin_password $SERVICE_PASSWORD iniset $SAHARA_CONF_FILE DEFAULT os_admin_username sahara iniset $SAHARA_CONF_FILE DEFAULT os_admin_tenant_name $SERVICE_TENANT_NAME # Set actual keystone auth configs iniset $SAHARA_CONF_FILE keystone_authtoken auth_uri $KEYSTONE_SERVICE_PROTOCOL://$KEYSTONE_SERVICE_HOST:$KEYSTONE_SERVICE_PORT/ + iniset $SAHARA_CONF_FILE keystone_authtoken auth_host $KEYSTONE_AUTH_HOST + iniset $SAHARA_CONF_FILE keystone_authtoken auth_port $KEYSTONE_AUTH_PORT + iniset $SAHARA_CONF_FILE keystone_authtoken auth_protocol $KEYSTONE_AUTH_PROTOCOL iniset $SAHARA_CONF_FILE keystone_authtoken admin_tenant_name $SERVICE_TENANT_NAME iniset $SAHARA_CONF_FILE keystone_authtoken admin_user sahara iniset $SAHARA_CONF_FILE keystone_authtoken admin_password $SERVICE_PASSWORD @@ -127,6 +132,8 @@ function configure_sahara { iniset $SAHARA_CONF_FILE DEFAULT verbose True iniset $SAHARA_CONF_FILE DEFAULT debug $ENABLE_DEBUG_LOG_LEVEL + iniset $SAHARA_CONF_FILE DEFAULT plugins $SAHARA_ENABLED_PLUGINS + iniset $SAHARA_CONF_FILE database connection `database_connection_url sahara` if is_service_enabled neutron; then @@ -142,6 +149,11 @@ function configure_sahara { iniset $SAHARA_CONF_FILE DEFAULT use_syslog $SYSLOG + # Format logging + if [ "$LOG_COLOR" == "True" ] && [ "$SYSLOG" == "False" ]; then + setup_colorized_logging $SAHARA_CONF_FILE DEFAULT + fi + recreate_database sahara utf8 $SAHARA_BIN_DIR/sahara-db-manage --config-file $SAHARA_CONF_FILE upgrade head } @@ -154,7 +166,7 @@ function install_sahara { # start_sahara() - Start running processes, including screen function start_sahara { - screen_it sahara "cd $SAHARA_DIR && $SAHARA_BIN_DIR/sahara-api --config-file $SAHARA_CONF_FILE" + screen_it sahara "cd $SAHARA_DIR && $SAHARA_BIN_DIR/sahara-all --config-file $SAHARA_CONF_FILE" } # stop_sahara() - Stop running processes diff --git a/lib/stackforge b/lib/stackforge index dca08cc2c2..e6528afc59 100644 --- a/lib/stackforge +++ b/lib/stackforge @@ -40,10 +40,10 @@ function install_stackforge { cleanup_stackforge git_clone $WSME_REPO $WSME_DIR $WSME_BRANCH - setup_develop_no_requirements_update $WSME_DIR + setup_package $WSME_DIR git_clone $PECAN_REPO $PECAN_DIR $PECAN_BRANCH - setup_develop_no_requirements_update $PECAN_DIR + setup_package $PECAN_DIR } # cleanup_stackforge() - purge possibly old versions of stackforge libraries diff --git a/lib/swift b/lib/swift index 3e183ff257..c47b09f6d7 100644 --- a/lib/swift +++ b/lib/swift @@ -103,6 +103,10 @@ SWIFT_REPLICAS_SEQ=$(seq ${SWIFT_REPLICAS}) # trace through the logs when looking for its use. SWIFT_LOG_TOKEN_LENGTH=${SWIFT_LOG_TOKEN_LENGTH:-12} +# Set ``SWIFT_MAX_HEADER_SIZE`` to configure the maximun length of headers in +# Swift API +SWIFT_MAX_HEADER_SIZE=${SWIFT_MAX_HEADER_SIZE:-16384} + # Set ``OBJECT_PORT_BASE``, ``CONTAINER_PORT_BASE``, ``ACCOUNT_PORT_BASE`` # Port bases used in port number calclution for the service "nodes" # The specified port number will be used, the additinal ports calculated by @@ -148,7 +152,7 @@ function _cleanup_swift_apache_wsgi { for type in object container account; do site_name=${type}-server-${node_number} disable_apache_site ${site_name} - sudo rm -f /etc/$APACHE_NAME/$APACHE_CONF_DIR/${site_name} + sudo rm -f $(apache_site_config_for ${site_name}) done done } @@ -156,18 +160,17 @@ function _cleanup_swift_apache_wsgi { # _config_swift_apache_wsgi() - Set WSGI config files of Swift function _config_swift_apache_wsgi { sudo mkdir -p ${SWIFT_APACHE_WSGI_DIR} - local apache_vhost_dir=/etc/${APACHE_NAME}/$APACHE_CONF_DIR local proxy_port=${SWIFT_DEFAULT_BIND_PORT:-8080} # copy proxy vhost and wsgi file - sudo cp ${SWIFT_DIR}/examples/apache2/proxy-server.template ${apache_vhost_dir}/proxy-server + sudo cp ${SWIFT_DIR}/examples/apache2/proxy-server.template $(apache_site_config_for proxy-server) sudo sed -e " /^#/d;/^$/d; s/%PORT%/$proxy_port/g; s/%SERVICENAME%/proxy-server/g; s/%APACHE_NAME%/${APACHE_NAME}/g; s/%USER%/${STACK_USER}/g; - " -i ${apache_vhost_dir}/proxy-server + " -i $(apache_site_config_for proxy-server) enable_apache_site proxy-server sudo cp ${SWIFT_DIR}/examples/wsgi/proxy-server.wsgi.template ${SWIFT_APACHE_WSGI_DIR}/proxy-server.wsgi @@ -182,13 +185,13 @@ function _config_swift_apache_wsgi { container_port=$[CONTAINER_PORT_BASE + 10 * ($node_number - 1)] account_port=$[ACCOUNT_PORT_BASE + 10 * ($node_number - 1)] - sudo cp ${SWIFT_DIR}/examples/apache2/object-server.template ${apache_vhost_dir}/object-server-${node_number} + sudo cp ${SWIFT_DIR}/examples/apache2/object-server.template $(apache_site_config_for object-server-${node_number}) sudo sed -e " s/%PORT%/$object_port/g; s/%SERVICENAME%/object-server-${node_number}/g; s/%APACHE_NAME%/${APACHE_NAME}/g; s/%USER%/${STACK_USER}/g; - " -i ${apache_vhost_dir}/object-server-${node_number} + " -i $(apache_site_config_for object-server-${node_number}) enable_apache_site object-server-${node_number} sudo cp ${SWIFT_DIR}/examples/wsgi/object-server.wsgi.template ${SWIFT_APACHE_WSGI_DIR}/object-server-${node_number}.wsgi @@ -197,14 +200,14 @@ function _config_swift_apache_wsgi { s/%SERVICECONF%/object-server\/${node_number}.conf/g; " -i ${SWIFT_APACHE_WSGI_DIR}/object-server-${node_number}.wsgi - sudo cp ${SWIFT_DIR}/examples/apache2/container-server.template ${apache_vhost_dir}/container-server-${node_number} + sudo cp ${SWIFT_DIR}/examples/apache2/container-server.template $(apache_site_config_for container-server-${node_number}) sudo sed -e " /^#/d;/^$/d; s/%PORT%/$container_port/g; s/%SERVICENAME%/container-server-${node_number}/g; s/%APACHE_NAME%/${APACHE_NAME}/g; s/%USER%/${STACK_USER}/g; - " -i ${apache_vhost_dir}/container-server-${node_number} + " -i $(apache_site_config_for container-server-${node_number}) enable_apache_site container-server-${node_number} sudo cp ${SWIFT_DIR}/examples/wsgi/container-server.wsgi.template ${SWIFT_APACHE_WSGI_DIR}/container-server-${node_number}.wsgi @@ -213,14 +216,14 @@ function _config_swift_apache_wsgi { s/%SERVICECONF%/container-server\/${node_number}.conf/g; " -i ${SWIFT_APACHE_WSGI_DIR}/container-server-${node_number}.wsgi - sudo cp ${SWIFT_DIR}/examples/apache2/account-server.template ${apache_vhost_dir}/account-server-${node_number} + sudo cp ${SWIFT_DIR}/examples/apache2/account-server.template $(apache_site_config_for account-server-${node_number}) sudo sed -e " /^#/d;/^$/d; s/%PORT%/$account_port/g; s/%SERVICENAME%/account-server-${node_number}/g; s/%APACHE_NAME%/${APACHE_NAME}/g; s/%USER%/${STACK_USER}/g; - " -i ${apache_vhost_dir}/account-server-${node_number} + " -i $(apache_site_config_for account-server-${node_number}) enable_apache_site account-server-${node_number} sudo cp ${SWIFT_DIR}/examples/wsgi/account-server.wsgi.template ${SWIFT_APACHE_WSGI_DIR}/account-server-${node_number}.wsgi @@ -404,6 +407,7 @@ EOF cp ${SWIFT_DIR}/etc/swift.conf-sample ${SWIFT_CONF_DIR}/swift.conf iniset ${SWIFT_CONF_DIR}/swift.conf swift-hash swift_hash_path_suffix ${SWIFT_HASH} + iniset ${SWIFT_CONF_DIR}/swift.conf swift-constraints max_header_size ${SWIFT_MAX_HEADER_SIZE} for node_number in ${SWIFT_REPLICAS_SEQ}; do swift_node_config=${SWIFT_CONF_DIR}/object-server/${node_number}.conf @@ -453,10 +457,13 @@ EOF rm -rf ${swift_log_dir} mkdir -p ${swift_log_dir}/hourly sudo chown -R ${STACK_USER}:adm ${swift_log_dir} - sed "s,%SWIFT_LOGDIR%,${swift_log_dir}," $FILES/swift/rsyslog.conf | sudo \ - tee /etc/rsyslog.d/10-swift.conf - # restart syslog to take the changes - sudo killall -HUP rsyslogd + + if [[ $SYSLOG != "False" ]]; then + sed "s,%SWIFT_LOGDIR%,${swift_log_dir}," $FILES/swift/rsyslog.conf | sudo \ + tee /etc/rsyslog.d/10-swift.conf + # restart syslog to take the changes + sudo killall -HUP rsyslogd + fi if is_apache_enabled_service swift; then _config_swift_apache_wsgi diff --git a/lib/tempest b/lib/tempest index a4558cedc1..6bb6109e13 100644 --- a/lib/tempest +++ b/lib/tempest @@ -28,7 +28,6 @@ # - ``DEFAULT_INSTANCE_TYPE`` # - ``DEFAULT_INSTANCE_USER`` # - ``CINDER_MULTI_LVM_BACKEND`` -# - ``HEAT_CREATE_TEST_IMAGE`` # # ``stack.sh`` calls the entry points in this order: # @@ -56,7 +55,7 @@ BUILD_INTERVAL=1 BUILD_TIMEOUT=196 -BOTO_MATERIALS_PATH="$FILES/images/s3-materials/cirros-0.3.1" +BOTO_MATERIALS_PATH="$FILES/images/s3-materials/cirros-${CIRROS_VERSION}" # Cinder/Volume variables TEMPEST_VOLUME_DRIVER=${TEMPEST_VOLUME_DRIVER:-default} @@ -65,6 +64,7 @@ TEMPEST_STORAGE_PROTOCOL=${TEMPEST_STORAGE_PROTOCOL:-iSCSI} # Neutron/Network variables IPV6_ENABLED=$(trueorfalse True $IPV6_ENABLED) +IPV6_SUBNET_ATTRIBUTES_ENABLED=$(trueorfalse True $IPV6_SUBNET_ATTRIBUTES_ENABLED) # Functions # --------- @@ -77,7 +77,6 @@ function configure_tempest { local num_images local image_uuid local image_uuid_alt - local errexit local password local line local flavors @@ -90,14 +89,6 @@ function configure_tempest { local boto_instance_type="m1.tiny" local ssh_connect_method="fixed" - # TODO(afazekas): - # sudo python setup.py deploy - - # This function exits on an error so that errors don't compound and you see - # only the first error that occurred. - errexit=$(set +o | grep errexit) - set -o errexit - # Save IFS ifs=$IFS @@ -153,6 +144,7 @@ function configure_tempest { # user and tenant are set up... ADMIN_USERNAME=${ADMIN_USERNAME:-admin} ADMIN_TENANT_NAME=${ADMIN_TENANT_NAME:-admin} + ADMIN_DOMAIN_NAME=${ADMIN_DOMAIN_NAME:-Default} TEMPEST_USERNAME=${TEMPEST_USERNAME:-demo} TEMPEST_TENANT_NAME=${TEMPEST_TENANT_NAME:-demo} ALT_USERNAME=${ALT_USERNAME:-alt_demo} @@ -250,9 +242,6 @@ function configure_tempest { iniset $TEMPEST_CONFIG compute build_timeout $BUILD_TIMEOUT iniset $TEMPEST_CONFIG volume build_timeout $BUILD_TIMEOUT iniset $TEMPEST_CONFIG boto build_timeout $BUILD_TIMEOUT - iniset $TEMPEST_CONFIG compute build_interval $BUILD_INTERVAL - iniset $TEMPEST_CONFIG volume build_interval $BUILD_INTERVAL - iniset $TEMPEST_CONFIG boto build_interval $BUILD_INTERVAL iniset $TEMPEST_CONFIG boto http_socket_timeout 5 # Identity @@ -267,6 +256,7 @@ function configure_tempest { iniset $TEMPEST_CONFIG identity admin_username $ADMIN_USERNAME iniset $TEMPEST_CONFIG identity admin_password "$password" iniset $TEMPEST_CONFIG identity admin_tenant_name $ADMIN_TENANT_NAME + iniset $TEMPEST_CONFIG identity admin_domain_name $ADMIN_DOMAIN_NAME iniset $TEMPEST_CONFIG identity auth_version ${TEMPEST_AUTH_VERSION:-v2} # Image @@ -308,32 +298,47 @@ function configure_tempest { iniset $TEMPEST_CONFIG network public_router_id "$public_router_id" iniset $TEMPEST_CONFIG network default_network "$FIXED_RANGE" iniset $TEMPEST_CONFIG network-feature-enabled ipv6 "$IPV6_ENABLED" + iniset $TEMPEST_CONFIG network-feature-enabled ipv6_subnet_attributes "$IPV6_SUBNET_ATTRIBUTES_ENABLED" # boto iniset $TEMPEST_CONFIG boto ec2_url "http://$SERVICE_HOST:8773/services/Cloud" iniset $TEMPEST_CONFIG boto s3_url "http://$SERVICE_HOST:${S3_SERVICE_PORT:-3333}" iniset $TEMPEST_CONFIG boto s3_materials_path "$BOTO_MATERIALS_PATH" - iniset $TEMPEST_CONFIG boto ari_manifest cirros-0.3.1-x86_64-initrd.manifest.xml - iniset $TEMPEST_CONFIG boto ami_manifest cirros-0.3.1-x86_64-blank.img.manifest.xml - iniset $TEMPEST_CONFIG boto aki_manifest cirros-0.3.1-x86_64-vmlinuz.manifest.xml + iniset $TEMPEST_CONFIG boto ari_manifest cirros-${CIRROS_VERSION}-x86_64-initrd.manifest.xml + iniset $TEMPEST_CONFIG boto ami_manifest cirros-${CIRROS_VERSION}-x86_64-blank.img.manifest.xml + iniset $TEMPEST_CONFIG boto aki_manifest cirros-${CIRROS_VERSION}-x86_64-vmlinuz.manifest.xml iniset $TEMPEST_CONFIG boto instance_type "$boto_instance_type" iniset $TEMPEST_CONFIG boto http_socket_timeout 30 iniset $TEMPEST_CONFIG boto ssh_user ${DEFAULT_INSTANCE_USER:-cirros} - # Orchestration test image - if [[ ! -z "$HEAT_FETCHED_TEST_IMAGE" ]]; then - iniset $TEMPEST_CONFIG orchestration image_ref "$HEAT_FETCHED_TEST_IMAGE" - elif [[ "$HEAT_CREATE_TEST_IMAGE" = "True" ]]; then - disk_image_create /usr/share/tripleo-image-elements "vm fedora heat-cfntools" "i386" "fedora-vm-heat-cfntools-tempest" - iniset $TEMPEST_CONFIG orchestration image_ref "fedora-vm-heat-cfntools-tempest" + # Orchestration Tests + if is_service_enabled heat; then + if [[ ! -z "$HEAT_CFN_IMAGE_URL" ]]; then + iniset $TEMPEST_CONFIG orchestration image_ref $(basename "$HEAT_CFN_IMAGE_URL" ".qcow2") + fi + # build a specialized heat flavor + available_flavors=$(nova flavor-list) + if [[ ! ( $available_flavors =~ 'm1.heat' ) ]]; then + nova flavor-create m1.heat 451 512 0 1 + fi + iniset $TEMPEST_CONFIG orchestration instance_type "m1.heat" + iniset $TEMPEST_CONFIG orchestration build_timeout 900 fi # Scenario - iniset $TEMPEST_CONFIG scenario img_dir "$FILES/images/cirros-0.3.1-x86_64-uec" + iniset $TEMPEST_CONFIG scenario img_dir "$FILES/images/cirros-${CIRROS_VERSION}-x86_64-uec" + iniset $TEMPEST_CONFIG scenario ami_img_file "cirros-${CIRROS_VERSION}-x86_64-blank.img" + iniset $TEMPEST_CONFIG scenario ari_img_file "cirros-${CIRROS_VERSION}-x86_64-initrd" + iniset $TEMPEST_CONFIG scenario aki_img_file "cirros-${CIRROS_VERSION}-x86_64-vmlinuz" # Large Ops Number iniset $TEMPEST_CONFIG scenario large_ops_number ${TEMPEST_LARGE_OPS_NUMBER:-0} + # Telemetry + # Ceilometer API optimization happened in juno that allows to run more tests in tempest. + # Once Tempest retires support for icehouse this flag can be removed. + iniset $TEMPEST_CONFIG telemetry too_slow_to_test "False" + # Volume if ! is_service_enabled c-bak; then iniset $TEMPEST_CONFIG volume-feature-enabled backup False @@ -360,6 +365,11 @@ function configure_tempest { # Networking iniset $TEMPEST_CONFIG network-feature-enabled api_extensions "${NETWORK_API_EXTENSIONS:-all}" + # Baremetal + if [ "$VIRT_DRIVER" = "ironic" ] ; then + iniset $TEMPEST_CONFIG baremetal driver_enabled True + fi + # service_available for service in ${TEMPEST_SERVICES//,/ }; do if is_service_enabled $service ; then @@ -371,8 +381,6 @@ function configure_tempest { # Restore IFS IFS=$ifs - #Restore errexit - $errexit } # create_tempest_accounts() - Set up common required tempest accounts @@ -402,12 +410,13 @@ function create_tempest_accounts { # install_tempest() - Collect source and prepare function install_tempest { git_clone $TEMPEST_REPO $TEMPEST_DIR $TEMPEST_BRANCH + pip_install "tox<1.7" } # init_tempest() - Initialize ec2 images function init_tempest { - local base_image_name=cirros-0.3.1-x86_64 - # /opt/stack/devstack/files/images/cirros-0.3.1-x86_64-uec + local base_image_name=cirros-${CIRROS_VERSION}-x86_64 + # /opt/stack/devstack/files/images/cirros-${CIRROS_VERSION}-x86_64-uec local image_dir="$FILES/images/${base_image_name}-uec" local kernel="$image_dir/${base_image_name}-vmlinuz" local ramdisk="$image_dir/${base_image_name}-initrd" @@ -419,9 +428,9 @@ function init_tempest { ( #new namespace # tenant:demo ; user: demo source $TOP_DIR/accrc/demo/demo - euca-bundle-image -i "$kernel" --kernel true -d "$BOTO_MATERIALS_PATH" - euca-bundle-image -i "$ramdisk" --ramdisk true -d "$BOTO_MATERIALS_PATH" - euca-bundle-image -i "$disk_image" -d "$BOTO_MATERIALS_PATH" + euca-bundle-image -r x86_64 -i "$kernel" --kernel true -d "$BOTO_MATERIALS_PATH" + euca-bundle-image -r x86_64 -i "$ramdisk" --ramdisk true -d "$BOTO_MATERIALS_PATH" + euca-bundle-image -r x86_64 -i "$disk_image" -d "$BOTO_MATERIALS_PATH" ) 2>&1 > $localrc @@ -425,7 +436,7 @@ initialize_database_backends && echo "Using $DATABASE_TYPE database backend" || # Rabbit connection info if is_service_enabled rabbit; then - RABBIT_HOST=${RABBIT_HOST:-localhost} + RABBIT_HOST=${RABBIT_HOST:-$SERVICE_HOST} read_password RABBIT_PASSWORD "ENTER A PASSWORD TO USE FOR RABBIT." fi @@ -483,14 +494,18 @@ function spinner { done } +function kill_spinner { + if [ ! -z "$LAST_SPINNER_PID" ]; then + kill >/dev/null 2>&1 $LAST_SPINNER_PID + printf "\b\b\bdone\n" >&3 + fi +} + # Echo text to the log file, summary log file and stdout # echo_summary "something to say" function echo_summary { if [[ -t 3 && "$VERBOSE" != "True" ]]; then - kill >/dev/null 2>&1 $LAST_SPINNER_PID - if [ ! -z "$LAST_SPINNER_PID" ]; then - printf "\b\b\bdone\n" >&3 - fi + kill_spinner echo -n -e $@ >&6 spinner & LAST_SPINNER_PID=$! @@ -528,29 +543,19 @@ if [[ -n "$LOGFILE" ]]; then # Redirect output according to config - # Copy stdout to fd 3 + # Set fd 3 to a copy of stdout. So we can set fd 1 without losing + # stdout later. exec 3>&1 if [[ "$VERBOSE" == "True" ]]; then - # Redirect stdout/stderr to tee to write the log file - exec 1> >( awk -v logfile=${LOGFILE} ' - /((set \+o$)|xtrace)/ { next } - { - cmd ="date +\"%Y-%m-%d %H:%M:%S.%3N | \"" - cmd | getline now - close("date +\"%Y-%m-%d %H:%M:%S.%3N | \"") - sub(/^/, now) - print > logfile - fflush(logfile) - print - fflush("") - }' ) 2>&1 - # Set up a second fd for output - exec 6> >( tee "${SUMFILE}" ) + # Set fd 1 and 2 to write the log file + exec 1> >( ./tools/outfilter.py -v -o "${LOGFILE}" ) 2>&1 + # Set fd 6 to summary log file + exec 6> >( ./tools/outfilter.py -o "${SUMFILE}" ) else # Set fd 1 and 2 to primary logfile - exec 1> "${LOGFILE}" 2>&1 + exec 1> >( ./tools/outfilter.py -o "${LOGFILE}" ) 2>&1 # Set fd 6 to summary logfile and stdout - exec 6> >( tee "${SUMFILE}" >&3 ) + exec 6> >( ./tools/outfilter.py -v -o "${SUMFILE}" >&3 ) fi echo_summary "stack.sh log $LOGFILE" @@ -559,14 +564,15 @@ if [[ -n "$LOGFILE" ]]; then ln -sf $SUMFILE $LOGDIR/$LOGFILENAME.summary else # Set up output redirection without log files - # Copy stdout to fd 3 + # Set fd 3 to a copy of stdout. So we can set fd 1 without losing + # stdout later. exec 3>&1 if [[ "$VERBOSE" != "True" ]]; then # Throw away stdout and stderr exec 1>/dev/null 2>&1 fi # Always send summary fd to original stdout - exec 6>&3 + exec 6> >( ./tools/outfilter.py -v >&3 ) fi # Set up logging of screen windows @@ -601,6 +607,10 @@ function exit_trap { echo "exit_trap: cleaning up child processes" kill 2>&1 $jobs fi + + # Kill the last spinner process + kill_spinner + exit $r } @@ -1001,9 +1011,13 @@ if is_service_enabled n-net q-dhcp; then fi clean_iptables - rm -rf ${NOVA_STATE_PATH}/networks - sudo mkdir -p ${NOVA_STATE_PATH}/networks - safe_chown -R ${USER} ${NOVA_STATE_PATH}/networks + + if is_service_enabled n-net; then + rm -rf ${NOVA_STATE_PATH}/networks + sudo mkdir -p ${NOVA_STATE_PATH}/networks + safe_chown -R ${STACK_USER} ${NOVA_STATE_PATH}/networks + fi + # Force IP forwarding on, just in case sudo sysctl -w net.ipv4.ip_forward=1 fi diff --git a/stackrc b/stackrc index 4418be1c06..01d0e0f79f 100644 --- a/stackrc +++ b/stackrc @@ -21,9 +21,9 @@ fi # Specify which services to launch. These generally correspond to # screen tabs. To change the default list, use the ``enable_service`` and -# ``disable_service`` functions in ``localrc``. -# For example, to enable Swift add this to ``localrc``: -# enable_service swift +# ``disable_service`` functions in ``local.conf``. +# For example, to enable Swift add this to ``local.conf``: +# enable_service s-proxy s-object s-container s-account # In order to enable Neutron (a single node setup) add the following # settings in `` localrc``: # disable_service n-net @@ -32,7 +32,6 @@ fi # enable_service q-dhcp # enable_service q-l3 # enable_service q-meta -# enable_service neutron # # Optional, to enable tempest configuration as part of devstack # enable_service tempest @@ -170,6 +169,10 @@ CLIFF_BRANCH=${CLIFF_BRANCH:-master} OSLOCFG_REPO=${OSLOCFG_REPO:-${GIT_BASE}/openstack/oslo.config.git} OSLOCFG_BRANCH=${OSLOCFG_BRANCH:-master} +# oslo.db +OSLODB_REPO=${OSLODB_REPO:-${GIT_BASE}/openstack/oslo.db.git} +OSLODB_BRANCH=${OSLODB_BRANCH:-master} + # oslo.messaging OSLOMSG_REPO=${OSLOMSG_REPO:-${GIT_BASE}/openstack/oslo.messaging.git} OSLOMSG_BRANCH=${OSLOMSG_BRANCH:-master} @@ -306,12 +309,14 @@ esac # glance as a disk image. If it ends in .gz, it is uncompressed first. # example: # http://cloud-images.ubuntu.com/releases/precise/release/ubuntu-12.04-server-cloudimg-armel-disk1.img -# http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-rootfs.img.gz +# http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros-${CIRROS_VERSION}-x86_64-rootfs.img.gz # * OpenVZ image: # OpenVZ uses its own format of image, and does not support UEC style images #IMAGE_URLS="http://smoser.brickies.net/ubuntu/ttylinux-uec/ttylinux-uec-amd64-11.2_2.6.35-15_1.tar.gz" # old ttylinux-uec image -#IMAGE_URLS="http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-disk.img" # cirros full disk image +#IMAGE_URLS="http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros-${CIRROS_VERSION}-x86_64-disk.img" # cirros full disk image + +CIRROS_VERSION=${CIRROS_VERSION:-"0.3.2"} # Set default image based on ``VIRT_DRIVER`` and ``LIBVIRT_TYPE``, either of # which may be set in ``localrc``. Also allow ``DEFAULT_IMAGE_NAME`` and @@ -323,27 +328,41 @@ case "$VIRT_DRIVER" in libvirt) case "$LIBVIRT_TYPE" in lxc) # the cirros root disk in the uec tarball is empty, so it will not work for lxc - DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-0.3.1-x86_64-rootfs} - IMAGE_URLS=${IMAGE_URLS:-"http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-rootfs.img.gz"};; + DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-${CIRROS_VERSION}-x86_64-rootfs} + IMAGE_URLS=${IMAGE_URLS:-"http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros-${CIRROS_VERSION}-x86_64-rootfs.img.gz"};; *) # otherwise, use the uec style image (with kernel, ramdisk, disk) - DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-0.3.1-x86_64-uec} - IMAGE_URLS=${IMAGE_URLS:-"http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz"};; + DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-${CIRROS_VERSION}-x86_64-uec} + IMAGE_URLS=${IMAGE_URLS:-"http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros-${CIRROS_VERSION}-x86_64-uec.tar.gz"};; esac ;; vsphere) - DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-debian-2.6.32-i686} - IMAGE_URLS=${IMAGE_URLS:-"http://partnerweb.vmware.com/programs/vmdkimage/cirros-0.3.0-i386-disk.vmdk"};; + DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-0.3.2-i386-disk.vmdk} + IMAGE_URLS=${IMAGE_URLS:-"http://partnerweb.vmware.com/programs/vmdkimage/cirros-0.3.2-i386-disk.vmdk"};; xenserver) DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-0.3.0-x86_64-disk} IMAGE_URLS=${IMAGE_URLS:-"https://github.com/downloads/citrix-openstack/warehouse/cirros-0.3.0-x86_64-disk.vhd.tgz"};; *) # Default to Cirros with kernel, ramdisk and disk image - DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-0.3.1-x86_64-uec} - IMAGE_URLS=${IMAGE_URLS:-"http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz"};; + DEFAULT_IMAGE_NAME=${DEFAULT_IMAGE_NAME:-cirros-${CIRROS_VERSION}-x86_64-uec} + IMAGE_URLS=${IMAGE_URLS:-"http://download.cirros-cloud.net/${CIRROS_VERSION}/cirros-${CIRROS_VERSION}-x86_64-uec.tar.gz"};; esac -HEAT_FETCHED_TEST_IMAGE=${HEAT_FETCHED_TEST_IMAGE:-""} -if [[ "$HEAT_FETCHED_TEST_IMAGE" == "Fedora-i386-20-20131211.1-sda" ]]; then - IMAGE_URLS+=",https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/i386/$HEAT_FETCHED_TEST_IMAGE.qcow2" +# Use 64bit fedora image if heat is enabled +if [[ "$ENABLED_SERVICES" =~ 'h-api' ]]; then + case "$VIRT_DRIVER" in + libvirt|baremetal|ironic) + HEAT_CFN_IMAGE_URL=${HEAT_CFN_IMAGE_URL:-"https://dl.fedoraproject.org/pub/fedora/linux/releases/20/Images/x86_64/Fedora-x86_64-20-20131211.1-sda.qcow2"} + IMAGE_URLS+=",$HEAT_CFN_IMAGE_URL" + ;; + *) + ;; + esac +fi +# Staging Area for New Images, have them here for at least 24hrs for nodepool +# to cache them otherwise the failure rates in the gate are too high +PRECACHE_IMAGES=$(trueorfalse False $PRECACHE_IMAGES) +if [[ "$PRECACHE_IMAGES" == "True" ]]; then + # staging in update for nodepool + IMAGE_URLS+=",https://dl.fedoraproject.org/pub/fedora/linux/updates/20/Images/x86_64/Fedora-x86_64-20-20140407-sda.qcow2" fi # 10Gb default volume backing file size diff --git a/tools/build_docs.sh b/tools/build_docs.sh index c566e6356f..384b1fabb6 100755 --- a/tools/build_docs.sh +++ b/tools/build_docs.sh @@ -22,7 +22,7 @@ # -------- # Source repo/branch for DevStack -MASTER_REPO=${MASTER_REPO:-https://github.com/openstack-dev/devstack.git} +MASTER_REPO=${MASTER_REPO:-git://git.openstack.org/openstack-dev/devstack} MASTER_BRANCH=${MASTER_BRANCH:-master} # http://devstack.org is a GitHub gh-pages site in the https://github.com/cloudbuilders/devtack.git repo diff --git a/tools/create_userrc.sh b/tools/create_userrc.sh index 47da3341b8..a34ad287f8 100755 --- a/tools/create_userrc.sh +++ b/tools/create_userrc.sh @@ -34,7 +34,7 @@ Optional Arguments -P include password to the rc files; with -A it assume all users password is the same -A try with all user -u create files just for the specified user --C create user and tenant, the specifid tenant will be the user's tenant +-C create user and tenant, the specifid tenant will be the user's tenant -r when combined with -C and the (-u) user exists it will be the user's tenant role in the (-C)tenant (default: Member) -p password for the user --os-username @@ -62,8 +62,8 @@ ADDPASS="" # The services users usually in the service tenant. # rc files for service users, is out of scope. -# Supporting different tanent for services is out of scope. -SKIP_TENANT=",service," # tenant names are between commas(,) +# Supporting different tenant for services is out of scope. +SKIP_TENANT="service" MODE="" ROLE=Member USER_NAME="" @@ -126,15 +126,15 @@ fi export -n SERVICE_TOKEN SERVICE_ENDPOINT OS_SERVICE_TOKEN OS_SERVICE_ENDPOINT -EC2_URL=http://localhost:8773/service/Cloud -S3_URL=http://localhost:3333 - -ec2=`keystone endpoint-get --service ec2 | awk '/\|[[:space:]]*ec2.publicURL/ {print $4}'` -[ -n "$ec2" ] && EC2_URL=$ec2 - -s3=`keystone endpoint-get --service s3 | awk '/\|[[:space:]]*s3.publicURL/ {print $4}'` -[ -n "$s3" ] && S3_URL=$s3 +EC2_URL=$(openstack endpoint show -f value -c publicurl ec2) +if [[ -z $EC2_URL ]]; then + EC2_URL=http://localhost:8773/services/Cloud +fi +S3_URL=$(openstack endpoint show -f value -c publicurl s3) +if [[ -z $S3_URL ]]; then + S3_URL=http://localhost:3333 +fi mkdir -p "$ACCOUNT_DIR" ACCOUNT_DIR=`readlink -f "$ACCOUNT_DIR"` @@ -158,13 +158,13 @@ function add_entry { local user_passwd=$5 # The admin user can see all user's secret AWS keys, it does not looks good - local line=`keystone ec2-credentials-list --user_id $user_id | grep -E "^\\|[[:space:]]*($tenant_name|$tenant_id)[[:space:]]*\\|" | head -n 1` + local line=`openstack ec2 credentials list --user $user_id | grep " $tenant_id "` if [ -z "$line" ]; then - keystone ec2-credentials-create --user-id $user_id --tenant-id $tenant_id 1>&2 - line=`keystone ec2-credentials-list --user_id $user_id | grep -E "^\\|[[:space:]]*($tenant_name|$tenant_id)[[:space:]]*\\|" | head -n 1` + openstack ec2 credentials create --user $user_id --project $tenant_id 1>&2 + line=`openstack ec2 credentials list --user $user_id | grep " $tenant_id "` fi local ec2_access_key ec2_secret_key - read ec2_access_key ec2_secret_key <<< `echo $line | awk '{print $4 " " $6 }'` + read ec2_access_key ec2_secret_key <<< `echo $line | awk '{print $2 " " $4 }'` mkdir -p "$ACCOUNT_DIR/$tenant_name" local rcfile="$ACCOUNT_DIR/$tenant_name/$user_name" # The certs subject part are the tenant ID "dash" user ID, but the CN should be the first part of the DN @@ -212,41 +212,35 @@ EOF } #admin users expected -function create_or_get_tenant { - local tenant_name=$1 - local tenant_id=`keystone tenant-list | awk '/\|[[:space:]]*'"$tenant_name"'[[:space:]]*\|.*\|/ {print $2}'` - if [ -n "$tenant_id" ]; then - echo $tenant_id - else - keystone tenant-create --name "$tenant_name" | awk '/\|[[:space:]]*id[[:space:]]*\|.*\|/ {print $4}' +function create_or_get_project { + local name=$1 + local id + eval $(openstack project show -f shell -c id $name) + if [[ -z $id ]]; then + eval $(openstack project create -f shell -c id $name) fi + echo $id } function create_or_get_role { - local role_name=$1 - local role_id=`keystone role-list| awk '/\|[[:space:]]*'"$role_name"'[[:space:]]*\|/ {print $2}'` - if [ -n "$role_id" ]; then - echo $role_id - else - keystone role-create --name "$role_name" |awk '/\|[[:space:]]*id[[:space:]]*\|.*\|/ {print $4}' + local name=$1 + local id + eval $(openstack role show -f shell -c id $name) + if [[ -z $id ]]; then + eval $(openstack role create -f shell -c id $name) fi + echo $id } # Provides empty string when the user does not exists function get_user_id { - local user_name=$1 - keystone user-list | awk '/^\|[^|]*\|[[:space:]]*'"$user_name"'[[:space:]]*\|.*\|/ {print $2}' + openstack user list | grep " $1 " | cut -d " " -f2 } if [ $MODE != "create" ]; then -# looks like I can't ask for all tenant related to a specified user - for tenant_id_at_name in `keystone tenant-list | awk 'BEGIN {IGNORECASE = 1} /true[[:space:]]*\|$/ {print $2 "@" $4}'`; do - read tenant_id tenant_name <<< `echo "$tenant_id_at_name" | sed 's/@/ /'` - if echo $SKIP_TENANT| grep -q ",$tenant_name,"; then - continue; - fi - for user_id_at_name in `keystone user-list --tenant-id $tenant_id | awk 'BEGIN {IGNORECASE = 1} /true[[:space:]]*\|[^|]*\|$/ {print $2 "@" $4}'`; do - read user_id user_name <<< `echo "$user_id_at_name" | sed 's/@/ /'` +# looks like I can't ask for all tenant related to a specified user + openstack project list --long --quote none -f csv | grep ',True' | grep -v "${SKIP_TENANT}" | while IFS=, read tenant_id tenant_name desc enabled; do + openstack user list --project $tenant_id --long --quote none -f csv | grep ',True' | while IFS=, read user_id user_name project email enabled; do if [ $MODE = one -a "$user_name" != "$USER_NAME" ]; then continue; fi @@ -263,18 +257,16 @@ if [ $MODE != "create" ]; then done else tenant_name=$TENANT - tenant_id=`create_or_get_tenant "$TENANT"` + tenant_id=$(create_or_get_project "$TENANT") user_name=$USER_NAME user_id=`get_user_id $user_name` if [ -z "$user_id" ]; then - #new user - user_id=`keystone user-create --name "$user_name" --tenant-id "$tenant_id" --pass "$USER_PASS" --email "$user_name@example.com" | awk '/\|[[:space:]]*id[[:space:]]*\|.*\|/ {print $4}'` - #The password is in the cmd line. It is not a good thing + eval $(openstack user create "$user_name" --project "$tenant_id" --password "$USER_PASS" --email "$user_name@example.com" -f shell -c id) + user_id=$id add_entry "$user_id" "$user_name" "$tenant_id" "$tenant_name" "$USER_PASS" else - #new role - role_id=`create_or_get_role "$ROLE"` - keystone user-role-add --user-id "$user_id" --tenant-id "$tenant_id" --role-id "$role_id" + role_id=$(create_or_get_role "$ROLE") + openstack role add "$role_id" --user "$user_id" --project "$tenant_id" add_entry "$user_id" "$user_name" "$tenant_id" "$tenant_name" "$USER_PASS" fi fi diff --git a/tools/fixup_stuff.sh b/tools/fixup_stuff.sh index 7833278a12..f1dc76a0af 100755 --- a/tools/fixup_stuff.sh +++ b/tools/fixup_stuff.sh @@ -35,6 +35,30 @@ source $TOP_DIR/functions FILES=$TOP_DIR/files +# Keystone Port Reservation +# ------------------------- +# Reserve and prevent $KEYSTONE_AUTH_PORT and $KEYSTONE_AUTH_PORT_INT from +# being used as ephemeral ports by the system. The default(s) are 35357 and +# 35358 which are in the Linux defined ephemeral port range (in disagreement +# with the IANA ephemeral port range). This is a workaround for bug #1253482 +# where Keystone will try and bind to the port and the port will already be +# in use as an ephemeral port by another process. This places an explicit +# exception into the Kernel for the Keystone AUTH ports. +keystone_ports=${KEYSTONE_AUTH_PORT:-35357},${KEYSTONE_AUTH_PORT_INT:-35358} + +# Get any currently reserved ports, strip off leading whitespace +reserved_ports=$(sysctl net.ipv4.ip_local_reserved_ports | awk -F'=' '{print $2;}' | sed 's/^ //') + +if [[ -z "${reserved_ports}" ]]; then + # If there are no currently reserved ports, reserve the keystone ports + sudo sysctl -w net.ipv4.ip_local_reserved_ports=${keystone_ports} +else + # If there are currently reserved ports, keep those and also reserve the + # keystone specific ports. Duplicate reservations are merged into a single + # reservation (or range) automatically by the kernel. + sudo sysctl -w net.ipv4.ip_local_reserved_ports=${keystone_ports},${reserved_ports} +fi + # Python Packages # --------------- @@ -87,16 +111,18 @@ if [[ ${DISTRO} =~ (precise) ]]; then fi -# RHEL6 -# ----- - -if [[ $DISTRO =~ (rhel6) ]]; then - +if is_fedora; then # Disable selinux to avoid configuring to allow Apache access # to Horizon files (LP#1175444) if selinuxenabled; then sudo setenforce 0 fi +fi + +# RHEL6 +# ----- + +if [[ $DISTRO =~ (rhel6) ]]; then # If the ``dbus`` package was installed by DevStack dependencies the # uuid may not be generated because the service was never started (PR#598200), @@ -145,4 +171,11 @@ if [[ $DISTRO =~ (rhel6) ]]; then # work unmolested. sudo ln -sf /usr/bin/nosetests1.1 /usr/local/bin/nosetests + # workaround for https://code.google.com/p/unittest-ext/issues/detail?id=79 + install_package python-unittest2 patch + pip_install discover + (cd /usr/lib/python2.6/site-packages/; sudo patch <"$FILES/patches/unittest2-discover.patch" || echo 'Assume already applied') + # Make sure the discover.pyc is up to date + sudo rm /usr/lib/python2.6/site-packages/discover.pyc || true + sudo python -c 'import discover' fi diff --git a/tools/image_list.sh b/tools/image_list.sh new file mode 100755 index 0000000000..f9a4e2f518 --- /dev/null +++ b/tools/image_list.sh @@ -0,0 +1,36 @@ +#!/bin/bash + +# Keep track of the devstack directory +TOP_DIR=$(cd $(dirname "$0")/.. && pwd) + +source $TOP_DIR/functions + +# Possible virt drivers, if we have more, add them here. Always keep +# dummy in the end position to trigger the fall through case. +DRIVERS="openvz ironic libvirt vsphere xenserver dummy" + +# Extra variables to trigger getting additional images. +ENABLED_SERVICES=h-api +HEAT_FETCHED_TEST_IMAGE="Fedora-i386-20-20131211.1-sda" +PRECACHE_IMAGES=True + +# Loop over all the virt drivers and collect all the possible images +ALL_IMAGES="" +for driver in $DRIVERS; do + VIRT_DRIVER=$driver + URLS=$(source $TOP_DIR/stackrc && echo $IMAGE_URLS) + if [[ ! -z "$ALL_IMAGES" ]]; then + ALL_IMAGES+=, + fi + ALL_IMAGES+=$URLS +done + +# Make a nice list +echo $ALL_IMAGES | tr ',' '\n' | sort | uniq + +# Sanity check - ensure we have a minimum number of images +num=$(echo $ALL_IMAGES | tr ',' '\n' | sort | uniq | wc -l) +if [[ "$num" -lt 5 ]]; then + echo "ERROR: We only found $num images in $ALL_IMAGES, which can't be right." + exit 1 +fi diff --git a/tools/install_pip.sh b/tools/install_pip.sh index 1eb9e7a3f5..150faaa258 100755 --- a/tools/install_pip.sh +++ b/tools/install_pip.sh @@ -24,28 +24,8 @@ source $TOP_DIR/functions FILES=$TOP_DIR/files -# Handle arguments - -USE_GET_PIP=${USE_GET_PIP:-0} -INSTALL_PIP_VERSION=${INSTALL_PIP_VERSION:-"1.4.1"} -while [[ -n "$1" ]]; do - case $1 in - --force) - FORCE=1 - ;; - --pip-version) - INSTALL_PIP_VERSION="$2" - shift - ;; - --use-get-pip) - USE_GET_PIP=1; - ;; - esac - shift -done - -PIP_GET_PIP_URL=https://raw.github.com/pypa/pip/master/contrib/get-pip.py -PIP_TAR_URL=https://pypi.python.org/packages/source/p/pip/pip-$INSTALL_PIP_VERSION.tar.gz +PIP_GET_PIP_URL=https://bootstrap.pypa.io/get-pip.py +LOCAL_PIP="$FILES/$(basename $PIP_GET_PIP_URL)" GetDistro echo "Distro: $DISTRO" @@ -62,23 +42,13 @@ function get_versions { function install_get_pip { - if [[ ! -r $FILES/get-pip.py ]]; then - (cd $FILES; \ - curl -O $PIP_GET_PIP_URL; \ - ) + if [[ ! -r $LOCAL_PIP ]]; then + curl -o $LOCAL_PIP $PIP_GET_PIP_URL || \ + die $LINENO "Download of get-pip.py failed" fi - sudo -E python $FILES/get-pip.py + sudo -E python $LOCAL_PIP } -function install_pip_tarball { - if [[ ! -r $FILES/pip-$INSTALL_PIP_VERSION.tar.gz ]]; then - (cd $FILES; \ - curl -O $PIP_TAR_URL; \ - tar xvfz pip-$INSTALL_PIP_VERSION.tar.gz 1>/dev/null) - fi - (cd $FILES/pip-$INSTALL_PIP_VERSION; \ - sudo -E python setup.py install 1>/dev/null) -} # Show starting versions get_versions @@ -88,10 +58,8 @@ get_versions # Eradicate any and all system packages uninstall_package python-pip -if [[ "$USE_GET_PIP" == "1" ]]; then - install_get_pip -else - install_pip_tarball -fi +install_get_pip + +pip_install -U setuptools get_versions diff --git a/tools/ironic/scripts/configure-vm b/tools/ironic/scripts/configure-vm index 9936b76c4f..4c42c491c5 100755 --- a/tools/ironic/scripts/configure-vm +++ b/tools/ironic/scripts/configure-vm @@ -9,6 +9,25 @@ templatedir = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'templates') +CONSOLE_LOG = """ + + + + + + + + + + + + + + + +""" + + def main(): parser = argparse.ArgumentParser( description="Configure a kvm virtual machine for the seed image.") @@ -30,6 +49,8 @@ def main(): help='The libvirt network name to use') parser.add_argument('--libvirt-nic-driver', default='e1000', help='The libvirt network driver to use') + parser.add_argument('--console-log', + help='File to log console') parser.add_argument('--emulator', default=None, help='Path to emulator bin for vm template') args = parser.parse_args() @@ -44,6 +65,7 @@ def main(): 'cpus': args.cpus, 'bootdev': args.bootdev, 'network': args.network, + 'nicdriver': args.libvirt_nic_driver, 'emulator': args.emulator, } @@ -55,22 +77,13 @@ def main(): elif os.path.exists("/usr/bin/qemu-kvm"): # Redhat params['emulator'] = "/usr/bin/qemu-kvm" - nicparams = { - 'nicdriver': args.libvirt_nic_driver, - 'network': args.network, - } - - params['bm_network'] = """ - - - - - -
-""" % nicparams - + if args.console_log: + params['console_log'] = CONSOLE_LOG % {'console_log': args.console_log} + else: + params['console_log'] = '' libvirt_template = source_template % params conn = libvirt.open("qemu:///system") + a = conn.defineXML(libvirt_template) print ("Created machine %s with UUID %s" % (args.name, a.UUIDString())) diff --git a/tools/ironic/scripts/create-nodes b/tools/ironic/scripts/create-nodes index d81113a4d5..140bffe46f 100755 --- a/tools/ironic/scripts/create-nodes +++ b/tools/ironic/scripts/create-nodes @@ -4,7 +4,7 @@ # Creates baremetal poseur nodes for ironic testing purposes -set -exu +set -ex # Keep track of the devstack directory TOP_DIR=$(cd $(dirname "$0")/.. && pwd) @@ -24,6 +24,7 @@ esac TOTAL=$(($5 - 1)) BRIDGE=$6 EMULATOR=$7 +LOGDIR=$8 LIBVIRT_NIC_DRIVER=${LIBVIRT_NIC_DRIVER:-"e1000"} LIBVIRT_STORAGE_POOL=${LIBVIRT_STORAGE_POOL:-"default"} @@ -43,6 +44,10 @@ if [ "$pool_state" != "running" ] ; then virsh pool-start $LIBVIRT_STORAGE_POOL >&2 fi +if [ -n "$LOGDIR" ] ; then + mkdir -p "$LOGDIR" +fi + PREALLOC= if [ -f /etc/debian_version ]; then PREALLOC="--prealloc-metadata" @@ -51,6 +56,11 @@ fi DOMS="" for idx in $(seq 0 $TOTAL) ; do NAME="baremetal${BRIDGE}_${idx}" + if [ -n "$LOGDIR" ] ; then + VM_LOGGING="--console-log $LOGDIR/${NAME}_console.log" + else + VM_LOGGING="" + fi DOMS="$DOMS $NAME" VOL_NAME="baremetal${BRIDGE}-${idx}.qcow2" (virsh list --all | grep -q $NAME) && continue @@ -62,7 +72,10 @@ for idx in $(seq 0 $TOTAL) ; do # Pre-touch the VM to set +C, as it can only be set on empty files. sudo touch "$volume_path" sudo chattr +C "$volume_path" || true - $TOP_DIR/scripts/configure-vm --bootdev network --name $NAME --image "$volume_path" --arch $ARCH --cpus $CPU --memory $MEM --libvirt-nic-driver $LIBVIRT_NIC_DRIVER --emulator $EMULATOR --network $BRIDGE >&2 + $TOP_DIR/scripts/configure-vm \ + --bootdev network --name $NAME --image "$volume_path" \ + --arch $ARCH --cpus $CPU --memory $MEM --libvirt-nic-driver $LIBVIRT_NIC_DRIVER \ + --emulator $EMULATOR --network $BRIDGE $VM_LOGGING >&2 done for dom in $DOMS ; do diff --git a/tools/ironic/templates/tftpd-xinetd.template b/tools/ironic/templates/tftpd-xinetd.template index 7b9b0f8a78..5f3d03f3bb 100644 --- a/tools/ironic/templates/tftpd-xinetd.template +++ b/tools/ironic/templates/tftpd-xinetd.template @@ -8,4 +8,7 @@ service tftp server = /usr/sbin/in.tftpd server_args = -v -v -v -v -v --map-file %TFTPBOOT_DIR%/map-file %TFTPBOOT_DIR% disable = no + # This is a workaround for Fedora, where TFTP will listen only on + # IPv6 endpoint, if IPv4 flag is not used. + flags = IPv4 } diff --git a/tools/ironic/templates/vm.xml b/tools/ironic/templates/vm.xml index b18dec055f..4f40334b7d 100644 --- a/tools/ironic/templates/vm.xml +++ b/tools/ironic/templates/vm.xml @@ -27,14 +27,19 @@
- %(network)s - %(bm_network)s + + + + +
+