.. _usersGuide_26_iterators: .. WARNING: DO NOT EDIT THIS FILE: AUTOMATICALLY GENERATED. PLEASE EDIT THE .py FILE DIRECTLY. User’s Guide, Chapter 26: Stream Iteration and Filtering ======================================================== We learned enough about streams in :ref:`Chapter 6 ` to be able to get started, but you’ve preservered and hopefully are ready to learn more about how to get the most out of getting through a score. So this chapter will delve deeper into the concept of iteration, that is, going through an object one step at a time, and filtering out elements so only those in classes or areas you want are found. Let’s review and describe the concept of iteration in Python (or most programming languages) for a second. Suppose you had a list like this: .. code:: ipython3 letterList = ['a', 'b', 'c'] Now you could get your ABCs out of it in this way: .. code:: ipython3 alphabet = '' alphabet += letterList[0] alphabet += letterList[1] alphabet += letterList[2] alphabet .. parsed-literal:: :class: ipython-result 'abc' But it’s far easier, especially for a big list, to *iterate* over it using a ``for`` loop: .. code:: ipython3 alphabet = '' for letter in letterList: alphabet += letter alphabet .. parsed-literal:: :class: ipython-result 'abc' We can *iterate* over a list because lists are *iterable* (or, conversely, for the tautology department, because we can *iterate* over a list, we call it *iterable*) and there are some functions and methods that do great things on iterable objects, such as join them: .. code:: ipython3 ''.join(letterList) .. parsed-literal:: :class: ipython-result 'abc' Or give the minimum value from a numeric list: .. code:: ipython3 min([10, 20, 30, -3423, 40]) .. parsed-literal:: :class: ipython-result -3423 Or give the length of an iterable: .. code:: ipython3 len(letterList) .. parsed-literal:: :class: ipython-result 3 In Python, there’s a special type of *iterable* object called a *generator* which gives out objects as they are needed. One generator that we have seen already is the ``range()`` function: .. code:: ipython3 zeroToFifty = range(51) zeroToFifty .. parsed-literal:: :class: ipython-result range(0, 51) We can find the first number in that range that is divisible by 5: .. code:: ipython3 for n in zeroToFifty: print(n) if n != 0 and n % 5 == 0: break .. parsed-literal:: :class: ipython-result 0 1 2 3 4 5 At this point we’ve stopped going through the ``range`` object and no more numbers are ever made or stored in memory – this point doesn’t matter to much for a set of numbers up to 50, but for numbers up to millions, or, as we will see, a repertory of scores of hundreds of thousands of notes, saving a few seconds here and there really adds up. Streams, as we have seen, are iterable: .. code:: ipython3 from music21 import * s = stream.Part(id='restyStream') s.append(note.Note('C#')) s.append(note.Rest(quarterLength=2.0)) s.append(note.Note('D', quarterLength=1.5)) s.append(note.Rest(quarterLength=1.0)) for thing in s: print(thing, thing.quarterLength) .. parsed-literal:: :class: ipython-result 1.0 2.0 1.5 1.0 When you iterate over a Stream, it is actually creating a lightweight object called a ``StreamIterator`` to help make things easier. We can create one directly by calling ``.iter()`` on any stream: .. code:: ipython3 sIter = s.iter() sIter .. parsed-literal:: :class: ipython-result .. note:: Prior to v.7, a `StreamIterator` was a property `.iter`. This information tells us that ``sIter`` is an iterator going over the ``Part`` object with id ``restyStream`` and it is currently ready to give out the first object, number 0. We can get the next thing in the Stream by calling ``next()`` on the Stream. .. code:: ipython3 next(sIter) .. parsed-literal:: :class: ipython-result .. code:: ipython3 next(sIter) .. parsed-literal:: :class: ipython-result .. code:: ipython3 sIter .. parsed-literal:: :class: ipython-result But for the most part, you’ll want to use the built in way of going through an iterable, that is, with a ``for`` loop: .. code:: ipython3 for el in sIter: print(el, el.quarterLength) .. parsed-literal:: :class: ipython-result 1.0 2.0 1.5 1.0 Filtering elements in iteration ------------------------------- So this does exactly what iterating directly on the Stream does – but it’s good to know that a ``StreamIterator`` is silently being generated so that you can see what else these Iterators do. Most importantly, a ``StreamIterator`` can add filters to it. Let’s add a ``ClassFilter`` from the :ref:`moduleStreamFilters` module: .. code:: ipython3 restFilter = stream.filters.ClassFilter('Rest') restIterator = sIter.addFilter(restFilter) for el in restIterator: print(el, el.quarterLength) .. parsed-literal:: :class: ipython-result 2.0 1.0 Now when we go through sIter, we are only getting those objects that match all of the filters on it. We can also filter by offset. Let’s create a new iterator and add an :class:`~music21.stream.filters.OffsetFilter` to it. .. code:: ipython3 sIter2 = s.iter() offsetFilter = stream.filters.OffsetFilter(offsetStart=0.5, offsetEnd=4.0) offsetIterator = sIter2.addFilter(offsetFilter) for el in offsetIterator: print(el, el.offset) .. parsed-literal:: :class: ipython-result 1.0 3.0 .. note:: prior to Music21 v.6, `sIter.addFilter()` would modify `sIter` in place and not return a new iterator. Thus in v.5, you would have written the last three lines of the code as: >>> sIter2.addFilter(offsetFilter) >>> for el in sIter2: ... print(el, el.offset) The changed behavior in v.6 did not affect most users, but it was one of the biggest backward incompatible changes -- it was worth breaking code to finally get this right. Multiple filters can be chained together to get something more powerful: .. code:: ipython3 for el in s.iter().addFilter(restFilter).addFilter(offsetFilter): print(el, el.offset) .. parsed-literal:: :class: ipython-result 1.0 Other filters that ``music21`` has in the :ref:`moduleStreamFilters` include: - :class:`~music21.stream.filters.IsFilter` which returns elements that are exactly the same as the objects passed in (useful for getting the context of an object in a stream) - :class:`~music21.stream.filters.IsNotFilter`, even more useful, for getting everything but an object or list of objects - :class:`~music21.stream.filters.IdFilter` for finding items by Id. - :class:`~music21.stream.filters.ClassNotFilter` for finding items other than a list of classes. - and :class:`~music21.stream.filters.GroupFilter` for finding elements which have a particular group name. Filter Shortcuts ~~~~~~~~~~~~~~~~ Filtering elements by offset or by class is so common, that ``music21`` has some shortcuts for adding filters to it, like this: .. code:: ipython3 sIter4 = s.iter() restIterator = sIter4.getElementsByClass('Rest') restOffsetIterator = restIterator.getElementsByOffset(0.5, 4.0) for el in restOffsetIterator: print(el, el.offset) .. parsed-literal:: :class: ipython-result 1.0 Easier still, since each of these methods returns a new filter object, you can chain them right in the for loop: .. code:: ipython3 for el in s.iter().getElementsByClass('Rest').getElementsByOffset(0.5, 4.0): print(el, el.offset) .. parsed-literal:: :class: ipython-result 1.0 And you can even skip the ``s.iter()`` step for getting an iterator for the most common of these filters, and ``music21`` will recognize what you want to do and create the iterator for you: .. code:: ipython3 for el in s.getElementsByClass('Rest').getElementsByOffset(0.5, 4.0): print(el, el.offset) .. parsed-literal:: :class: ipython-result 1.0 The shortcut methods that ``music21`` exposes on Iterators include: - :meth:`~music21.stream.iterator.StreamIterator.getElementById` which adds an ``IdFilter`` - :meth:`~music21.stream.iterator.StreamIterator.getElementsByClass` which adds a ``ClassFilter`` - :meth:`~music21.stream.iterator.StreamIterator.getElementsByGroup` which adds a ``GroupFilter`` - :meth:`~music21.stream.iterator.StreamIterator.getElementsByOffset` which adds an ``OffsetFilter`` And there are also properties (that is, written without parentheses) which add certain filters: - :attr:`~music21.stream.iterator.StreamIterator.notes` which filters out everything but ``Note`` and ``Chord`` objects - :attr:`~music21.stream.iterator.StreamIterator.notesAndRests` which filters out everything except ``GeneralNote`` objects - :attr:`~music21.stream.iterator.StreamIterator.parts` which returns all the ``Part`` objects - :attr:`~music21.stream.iterator.StreamIterator.voices` which returns all the ``Voice`` objects - :attr:`~music21.stream.iterator.StreamIterator.spanners` which returns all the ``Spanner`` objects Custom Filters ~~~~~~~~~~~~~~ Creating your own filter is pretty easy too. The easiest way is to create a function that takes in an element and returns True or False depending on whether the object matches the filter. We will create a filter to see if the element has a ``.pitch`` attribute and then if that pitch attribute has a sharp on it: .. code:: ipython3 def sharpFilter(el): if (hasattr(el, 'pitch') and el.pitch.accidental is not None and el.pitch.accidental.alter > 0): return True else: return False sharpIterator = s.iter().addFilter(sharpFilter) for el in sharpIterator: print(el) .. parsed-literal:: :class: ipython-result Recursive and Offset Iterators ------------------------------ ``Music21`` comes with two other iterators that let you do powerful operations. The most commonly used is the :class:`~music21.stream.iterators.RecursiveIterator` which burrows down into nested Streams to get whatever you want. Let’s load in a nested stream: .. code:: ipython3 bach = corpus.parse('bwv66.6') for thing in bach: print(thing) .. parsed-literal:: :class: ipython-result > Right, we remember that often the actual notes of a piece can be hidden inside Parts, Measures, and Voices. A recursive iterator gets to them, and they’re created by calling ``recurse()`` on a stream. .. code:: ipython3 recurseIter = bach.recurse() recurseIter .. parsed-literal:: :class: ipython-result Let’s add a filter for only E#s to it, and look into it. Instead of checking to see if each element has a ``.name`` attribute we’ll put a ``try...except`` clause around it, and if it does not have the ``.name`` attribute (and thus raises and ``AttributeError`` we will return False. .. code:: ipython3 def eSharpFilter(el): try: if el.name == 'E#': return True else: return False except AttributeError: return False eSharpIterator = recurseIter.addFilter(eSharpFilter) for el in eSharpIterator: print(el, el.measureNumber) .. parsed-literal:: :class: ipython-result 9 3 7 7 2 6 Note that the measure numbers don’t keep increasing. That’s because the recurse iterator finishes one part before returning to the next. We can use the fancy ``.getContextByClass`` to figure out what part it is in: .. code:: ipython3 for el in eSharpIterator: pId = el.getContextByClass(stream.Part).id print(el, el.measureNumber, pId) .. parsed-literal:: :class: ipython-result 9 Soprano 3 Alto 7 Alto 7 Tenor 2 Bass 6 Bass (as an aside, ``.measureNumber`` is just a shortcut for ``.getContextByClass(stream.Measure).number``, so we are actually looking up two contexts) If you want to recurse into a stream and get elements of a certain class, you can do ``s.recurse().getElementsByClass(chord.Chord)`` but there’s another simpler way of doing it: ``s[chord.Chord]`` (with square brackets). As this example shows: .. code:: ipython3 chopin = corpus.parse('chopin/mazurka06-2') chopinExcerpt = chopin.measures(1, 5) for ch in chopinExcerpt[chord.Chord]: print(ch) .. parsed-literal:: :class: ipython-result (when Chopin likes a chord, he **really** likes a chord!). Note that each of these is a chord in one voice in one hand of the piano. To see how to get chords between both hands, turn back to the ``chordify()`` chapter. Another great iterator is the OffsetIterator, which returns lists of elements grouped by offset. Let’s add some more things to our Stream before we see how it works. .. code:: ipython3 s.insert(0, clef.TrebleClef()) s.insert(0, key.KeySignature(3)) s.insert(1, instrument.Trumpet()) # normal iterator for el in s: print(el, el.offset) .. parsed-literal:: :class: ipython-result 0.0 0.0 0.0 Trumpet 1.0 1.0 3.0 4.5 Unlike with the normal ``StreamIterator`` or the ``RecursiveIterator``, there is no method on ``Stream`` to create an offset iterator, so we will create one directly: .. code:: ipython3 oIter = stream.iterator.OffsetIterator(s) for elementGroup in oIter: print(elementGroup[0].offset, elementGroup) .. parsed-literal:: :class: ipython-result 0.0 [, , ] 1.0 [, ] 3.0 [] 4.5 [] From Iterator to Stream ----------------------- From either a ``StreamIterator`` or a ``RecursiveIterator`` a new ``Stream`` object can be generated by calling ``.stream()`` on it. On a ``RecursiveIterator``, this does not put the elements into substreams. .. code:: ipython3 onlyESharps = bach.recurse().addFilter(eSharpFilter) esharpStream = onlyESharps.stream() esharpStream.show('text') .. parsed-literal:: :class: ipython-result {8.0} {10.0} {23.0} {25.5} {27.0} {34.5} .. code:: ipython3 esharpStream.derivation .. parsed-literal:: :class: ipython-result from via 'eSharpFilter'> This can be useful if you’d like to do plots on the resulting stream, though this one is a bit too obvious… .. code:: ipython3 esharpStream.plot('pitchclass') .. image:: usersGuide_26_iterators_66_0.png :width: 504px :height: 540px But maybe this one could tell someone something: .. code:: ipython3 esharpStream.plot('pianoroll') .. image:: usersGuide_26_iterators_68_0.png :width: 816px :height: 540px Perhaps not. But iterators are not the main point – what you can do with them is more important, so we will return to working with actual musical objects in :ref:`Chapter 27 ` when we talk about Grace Notes.