Robert Anasch viaUnsplash
Learn how you can parse, explore, modify and populate XML files with the python ElementTree package, for loops and XPath expressions. As a data scientist, you’ll find that understanding XML is powerful for both web-scraping and general practice in parsing a structured document
Extensible Markup Language(XML) is a markup language which encodes documents by defining a set of rules in both machine-readable and human-readable format. Extended from SGML (Standard Generalized Markup Language), it lets us describe the structure of the document. In XML, we can define custom tags. We can also use XML as a standard format to exchange information.
XML documents have sections, called elements , defined by a beginning and an ending tag . A tag is a markup construct that begins with < and ends with > . The characters between the start-tag and end-tag, if there are any, are the element's content. Elements can contain markup, including other elements, which are called "child elements". The largest, top-level element is called the root , which contains all other elements. Attributes are name value pair that exist within a start-tag or empty-element tag. An XML attribute can only have a single value and each attribute can appear at most once on each element.Here’s a snapshot of movies.xml that we will be using for this tutorial:
<?xml version="1.0"?> <collection> <genre category="Action"> <decade years="1980s"> <movie favorite="True" title="Indiana Jones: The raiders of the lost Ark"> <format multiple="No">DVD</format> <year>1981</year> <rating>PG</rating> <description> 'Archaeologist and adventurer Indiana Jones is hired by the U.S. government to find the Ark of the Covenant before the Nazis.' </description> </movie> <movie favorite="True" title="THE KARATE KID"> <format multiple="Yes">DVD,Online</format> <year>1984</year> <rating>PG</rating> <description>None provided.</description> </movie> <movie favorite="False" title="Back 2 the Future"> <format multiple="False">Blu-ray</format> <year>1985</year> <rating>PG</rating> <description>Marty McFly</description> </movie> </decade> <decade years="1990s"> <movie favorite="False" title="X-Men"> <format multiple="Yes">dvd, digital</format> <year>2000</year> <rating>PG-13</rating> <description>Two mutants come to a private academy for their kind whose resident superhero team must oppose a terrorist organization with similar powers.</description> </movie> <movie favorite="True" title="Batman Returns"> <format multiple="No">VHS</format> <year>1992</year> <rating>PG13</rating> <description>NA.</description> </movie> <movie favorite="False" title="Reservoir Dogs"> <format multiple="No">Online</format> <year>1992</year> <rating>R</rating> <description>WhAtEvER I Want!!!?!</description> </movie> </decade> </genre> <genre category="Thriller"> <decade years="1970s"> <movie favorite="False" title="ALIEN"> <format multiple="Yes">DVD</format> <year>1979</year> <rating>R</rating> <description>"""""""""</description> </movie> </decade> <decade years="1980s"> <movie favorite="True" title="Ferris Bueller's Day Off"> <format multiple="No">DVD</format> <year>1986</year> <rating>PG13</rating> <description>Funny movie on funny guy </description> </movie> <movie favorite="FALSE" title="American Psycho"> <format multiple="No">blue-ray</format> <year>2000</year> <rating>Unrated</rating> <description>psychopathic Bateman</description> </movie> </decade> </genre> Introduction to ElementTreeThe XML tree structure makes navigation, modification, and removal relatively simple programmatically. Python has a built in library, ElementTree , that has functions to read and manipulate XMLs (and other similarly structured files).
First, import ElementTree . It's a common practice to use the alias of ET :
<strong>import xml.etree.ElementTree as ET</strong> Parsing XMLDataIn the XML file provided, there is a basic collection of movies described. The only problem is the data is a mess! There have been a lot of different curators of this collection and everyone has their own way of entering data into the file. The main goal in this tutorial will be to read and understand the file with Python ― then fix the problems.
First you need to read in the file with ElementTree .
tree = ET.parse('movies.xml')root = tree.getroot()
Now that you have initialized the tree, you should look at the XML and print out values in order to understand how the tree is structured.
<strong>root.tag</strong> 'collection'At the top level, you see that this XML is rooted in the collection tag.
<strong>root.attrib</strong> {} For LoopsYou can easily iterate over subelements (commonly called “children”) in the root by using a simple “for” loop.
for child in root:print(child.tag, child.attrib) genre {'category': 'Action'} genre {'category': 'Thriller'} genre {'category': 'Comedy'}
Now you know that the children of the root collection are all genre . To designate the genre, the XML uses the attribute category . There are Action, Thriller, and Comedy movies according the genre element.
Typically it is helpful to know all the elements in the entire tree. One useful function for doing that is root.iter() .
<strong>[elem.tag for elem in root.iter()]</strong> ['collection', 'genre', 'decade', 'movie', 'format', 'year', 'rating', 'description', 'movie', . . . . 'movie', 'format', 'year', 'rating', 'description'] There is a helpful way to see the whole document. If you pass the root into the .tostring() method, you can return the whole document. Within
