SETA rewrite-Use SqlAlchemy to instead of hardcode sql

Due to we have already make SETA to support taskcluster now, the nextthing we should do is to make it make the code become more robust and readable. That’s it, we should add more tests and documents for it. The first step, before we start to write test for it, is no longer use hardcode sql in SETA. For example.

sql = """insert into testjobs (slave, result, duration, platform, buildtype, testtype, bugid, branch, revision, date, failure_classification, failures) values ('%s', '%s', %s, '%s', '%s', '%s', '%s', '%s', '%s', '%s', %s, '%s')""" % \ (slave, result, duration, platform, buildtype, testtype, bugid, branch, revision, date, failure_classification, ','.join(failures)

It makes us hard to write test for it and cased some bugs when we need to switch database in different environment(because we have master mysql branch and heroku postgresql branch). Therefore, the solution I found is use SqlAlchemy which is a powerful and useful database toolkit for python. You can found some more information about on its office website .

Alright, to useSqlAlchemy in SETA, we should make it connects to the database, which is create a engine for it:

engine = create_engine('mysql+mysqldb://root:root@localhost/ouija2', echo=False) The standard calling form is to send the URL as the first positional argument, usually a string that indicates database dialect and connection arguments. The string form of the URL is dialect[+driver]://user:password@host/dbname[?key=value..], where dialect is a database name such as mysql, oracle, postgresql, etc., and driver the name of a DBAPI, such as psycopg2, pyodbc, cx_oracle, etc. Alternatively, the URL can be an instance of URL.

And we need to create some database table for our new database. Classes mapped using the Declarative system are defined in terms of a base class which maintains a catalog of classes and tables relative to that base this is known as the declarative model. And we could use Metadata to include all the models we define and bind it with our database.

class Seta(MetaBase): __tablename__ = 'seta' id = Column(Integer, primary_key=True) jobtype = Column(String(256), nullable=False) date = Column(DateTime, nullable=False, index=True) def __init__(self, jobtype, date): self.jobtype = jobtype self.date = date

We’re now ready to start talking to the database. The ORM’s “handle” to the database is the Session. When we first set up the application, at the same level as our create_engine() statement, we define a Session class which will serve as a factory for new Session objects and use it to query or operate the database.

In config.py

Session = sessionmaker(engine) session = Session()

In failure.py(which is the file we need to use the database operation)

session.query(Seta).filter(Seta.date == date).delete(synchronize_session='fetch')

That’s it! And you can read some more detail in this PR . The next step for SETA rewrite is write more test for it!