Research talk:Revision scoring as a service/Work log/2016-03-25

Friday, March 25, 2016

Scap3 puppet settings for ORES

Hey. Today, after a month-long marathon we are getting really close to deploy ORES in production cluster. I did make wheels and also made mw:scpa3 config files. But scap3-related puppet configurations are still missing. I talked to releng people and after lots of discussions (phab:P2810) and lots of helps in IRC. We have two instances in beta, one for web and for worker deployment-ores-worker.deployment-prep.eqiad.wmflabs and deployment-ores-web.deployment-prep.eqiad.wmflabs. Then I tried running several puppet configs using puppet apply and not puppetmaster and I've got lots of issues and I tried to solve them one by one. This is my latest puppet script that we should add to our configs. (note: It'll affect ordinary web nodes in ORES project in labs, we should figure a way out or just let them have it!)

class {'scap':
  deployment_server => 'deployment-tin.deployment-prep.eqiad.wmflabs',
}

user { 'ladsgroup':
    ensure => present,
}

scap::target { 'ores/deploy':
    deploy_user => 'ladsgroup',
    public_key_source => "puppet:///modules/test/pub.pub",
    sudo_rules        => [
        'ALL=(root) NOPASSWD: /usr/sbin/service uwsgi-ores-web *',
        'ALL=(root) NOPASSWD: /usr/sbin/service celery-ores-worker *',
        'ALL=(root) NOPASSWD: /usr/sbin/service flower-ores *',
    ],
    manage_user => false,
}

1. In the first class declaration, I had to explicitly declare deployment_server (tin in beta) because it couldn't load hiera configs (and I even tried copying wikitech:Hiera:Deployment-prep in /etc/puppet/hiera.yaml. It loaded the hiera file but couldn't use it in scap class)
2. It would be great if we use a general user instead of me. I think Ops need to decide on that.
3. This puppet still doesn't work, simply because ores::base protects /srv/ores in root mode and my account doesn't have write access. We can give my user (or anyone's) a proper access and then we can move on to the next issue (which is unknown to me :D) Amir (talk) 00:22, 25 March 2016 (UTC)

OK. This puppet ran successfully in target and I was able to do deploy using scap3 in tin. Obviously, some settings would be redundant in prod (and using puppetmaster instead of "puppet apply"):

class {'scap':
  deployment_server => 'deployment-tin.deployment-prep.eqiad.wmflabs',
}

user { 'ladsgroup':
    ensure => present,
}

scap::target { 'ores/ores-wikimedia-config':
    deploy_user => 'ladsgroup',
    public_key_source => "puppet:///modules/test/pub.pub",
    sudo_rules        => [
        'ALL=(root) NOPASSWD: /usr/sbin/service uwsgi-ores-web *',
        'ALL=(root) NOPASSWD: /usr/sbin/service celery-ores-worker *',
        'ALL=(root) NOPASSWD: /usr/sbin/service flower-ores *',
    ],
    manage_user => false,
}

file { [
        '/srv/ores',
    ]:
        ensure  => directory,
        owner   => 'ladsgroup',
        group   => 'wikidev',
        mode    => '0775',
}

YayAmir (talk) 17:01, 25 March 2016 (UTC)