Streaming data

Streaming data#

  • Data that can be loaded chunk-wise (single or a few at the time).

  • Data flowing through a socket/pipe/streaming API/…

  • Typically requires:

    • Checking if stream is active.

    • A reactive reading mechanism or time delayed loop to check for updates.

    • Error/exception checking/handling.

    • A receiving enitity that can use the incomming data, e.g., printing to screen, updating graphics, updating a model, predicting some outcome/state/quantity, …

Live reading of Twitch chat#

# Check if user is logged in ("==" active chat)
import requests
user = "summit1g" # Change this to the user you want to check, e.g., epicdan22, zackrawrr, summit1g, 
# mizkif, cohhcarnage, k3soju, etc.
response = requests.get("https://decapi.me/twitch/uptime/"+user).text
is_online = response != user+" is offline"
print(is_online)

True

# Connect to the Twitch chat using River
from river import stream

if is_online:
    oauth = open('../../../No_sync/twitch_oauth','r').read()
    twitch_chat = stream.TwitchChatStream(
        nickname="khliland", # Exchange with your Twitch username
        token=oauth,
        channels=[user]
    )

# If the user is online, print the first 2 messages
if is_online:
    messages = 10
    for item in twitch_chat:
        print(item)
        if messages > 1:
            messages -= 1
        else:
            print("Puh, that's enough!")
            break

{'dt': datetime.datetime(2025, 11, 24, 9, 16, 13, 281031), 'channel': 'summit1g', 'username': 'rena9632', 'msg': '27 in April'}

{'dt': datetime.datetime(2025, 11, 24, 9, 16, 14, 127149), 'channel': 'summit1g', 'username': 'burberrybloodstains', 'msg': '“have you tried Stretching?”'}

{'dt': datetime.datetime(2025, 11, 24, 9, 16, 14, 642385), 'channel': 'summit1g', 'username': 'campifyz', 'msg': 'mhm'}
{'dt': datetime.datetime(2025, 11, 24, 9, 16, 14, 826454), 'channel': 'summit1g', 'username': 'joshie1k', 'msg': '@Niner9er 26 for 20 years straight LUL'}

{'dt': datetime.datetime(2025, 11, 24, 9, 16, 15, 10836), 'channel': 'summit1g', 'username': 'uhhmazing', 'msg': '@clipsbymercy what you want him to do about it lol?'}

{'dt': datetime.datetime(2025, 11, 24, 9, 16, 15, 305542), 'channel': 'summit1g', 'username': 'blazee_hazee', 'msg': 'LMAO'}

{'dt': datetime.datetime(2025, 11, 24, 9, 16, 16, 47403), 'channel': 'summit1g', 'username': 'fellownick', 'msg': 'he’s 37 chat google is free bros been 26 for yards'}
{'dt': datetime.datetime(2025, 11, 24, 9, 16, 16, 232640), 'channel': 'summit1g', 'username': 'ghilleh', 'msg': 'LOL'}

{'dt': datetime.datetime(2025, 11, 24, 9, 16, 17, 493406), 'channel': 'summit1g', 'username': 'fellownick', 'msg': 'years'}

{'dt': datetime.datetime(2025, 11, 24, 9, 16, 17, 969478), 'channel': 'summit1g', 'username': 'xblontwitch', 'msg': 'whaaa?!'}
Puh, that's enough!

Comments#

  • Here, River has formated everything nicely for us as dictionaries.

    • River’s streaming format is based on dictionaries to minimize overhead.

  • See escpecially the datetime formatting.

  • We will return to River and streaming Machine Learning later in the book.

Exercise#

Make a clock using TimeAPI.io

See also

Resources

Contents