TickerRain is an open-source web app that stores and analysis Reddit posts in a transparent and semi-interactive manner.

Overview

Tickerrain

TickerRain is an open-source web app that stores and analysis Reddit posts in a transparent and semi-interactive manner.

Overview

A simple webpage will display the sentimental analysis and entities of the last post processed, then it will display DB info and finally three graphs of the most mentioned tickers in Reddit.

Web server

The graphs are updated every 120 seconds and refreshing the page will display the analysis of a new post.

Requirements

Python3 and the following packages:

  • pandas
  • flask
  • redis
  • cairosvg
  • nltk
  • spacy
  • matplotlib
  • asyncpraw
  • cachetools

Other than that you need Cairo, for example, for Ubuntu run apt-get install libpangocairo-1.0-0.

Running

First, make sure you have a Redis DB running.

In the file substoscrap.txt specify what subreddits to analyze.

There are 3 parts, a process to get the submissions and store them in Redis DB, one to process them, and then finally one to run the webserver.

Getting Submissions

Run python news.py with arguments the crendentials for your account reddit API, see more here.

This will start getting posts, comments, and Redditors from Reddit and store them in Redis DB.

Processing Posts

Run python -m spacy download en_core_web_lgto get spacy-model required for processing posts.

Run python process.py, this will connect to the DB and start calculating metrics every 120 seconds, the results will be stored in 3 files, tickers_df_.p.

The metrics computed right now are:

  • Mentions -> Detectes what ticker is being talked about and counts the total mentions of it.
  • Score -> Calculates the log score which takes into account the upvotes and downvotes.
  • Sentiment -> Using Spacy Vader sentimental analysis it aggregates the general sentiment about the ticker.

Flask Web Server

Run python flask_example.py to start the webserver that displays the results, DB infos and the last post being processed. Access it by opening a browser and going to 127.0.0.1:5000

Issues and TODO

Currently, the processing code, using Pandas, needs to be optimized, it needs to use Pandas in a better way. The ticker detection needs to be improved, it emits warnings and misses some.

  • Improve ticker detecting, combining Spacy entities.
  • Optimize Pandas processing.
  • Add more metrics.
  • Improve the design of the Web page.
  • Auto download of tickers.csv from NASDAQ.
Owner
GonVas
https://www.linkedin.com/in/goncalomoreno/
GonVas
Simple and easy to use python API for the COVID registration booking system of the math department @ unipd (torre archimede)

Simple and easy to use python API for the COVID registration booking system of the math department @ unipd (torre archimede). This API creates an interface with the official browser, with more useful

Guglielmo Camporese 4 Dec 24, 2021
an opensourced roblox group finder writen in python 100% free and virus-free

Roblox-Group-Finder an opensourced roblox group finder writen in python 100% free and virus-free note : if you don't want install python or just use w

mollomm1 1 Nov 11, 2021
A collection of common regular expressions bundled with an easy to use interface.

CommonRegex Find all times, dates, links, phone numbers, emails, ip addresses, prices, hex colors, and credit card numbers in a string. We did the har

Madison May 1.5k Dec 31, 2022
Open Source Repository for CFD Solvers

Background and Validation This wiki is built in Notion. Here are all the tips you need to contribute. General Background Flow over a cylinder The proj

1 Dec 30, 2021
Double Pendulum implementation in Python, now with added pendulums and trails :D

Double Pendulum Using Curses in Python. A nice relaxing double pendulum simulation using ASCII, able to simulate multiple pendulums at once, and provi

Nekurone 62 Dec 14, 2022
Repository for DNN training, theory to practice, part of the Large Scale Machine Learning class at Mines Paritech

DNN Training, from theory to practice This repository is complementary to the deep learning training lesson given to les Mines ParisTech on the 11th o

Alexandre Défossez 6 Nov 14, 2022
A small Python library which gives you the IEEE-754 representation of a floating point number.

ieee754 ieee754 is small Python library which gives you the IEEE-754 representation of a floating point number. You can specify a precision given in t

Bora Canbula 5 Dec 20, 2022
Small tool to use hero .json files created with Optolith for The Dark Eye/ Das Schwarze Auge 5 to perform talent probes.

DSA5-ProbeMaker A little tool for The Dark Eye 5th Edition (Das Schwarze Auge 5) to load .json from Optolith character generation and easily perform t

2 Jan 06, 2022
Test to grab m3u from YouTube live.

YouTube_to_m3u https://raw.githubusercontent.com/benmoose39/YouTube_to_m3u/main/youtube.m3u Updated m3u links of YouTube live channels, auto-updated e

136 Jan 06, 2023
MDAnalysis tool to calculate membrane curvature.

The MDAkit for membrane curvature analysis is part of the Google Summer of Code program and it is linked to a Code of Conduct.

MDAnalysis 19 Oct 20, 2022
Simple yet flexible natural sorting in Python.

natsort Simple yet flexible natural sorting in Python. Source Code: https://github.com/SethMMorton/natsort Downloads: https://pypi.org/project/natsort

Seth Morton 712 Dec 23, 2022
You'll learn about Iterators, Generators, Closure, Decorators, Property, and RegEx in detail with examples.

07_Python_Advanced_Topics Introduction 👋 In this tutorial, you will learn about: Python Iterators: They are objects that can be iterated upon. In thi

Milaan Parmar / Милан пармар / _米兰 帕尔马 252 Dec 23, 2022
Machine Learning powered app to decide whether a photo is food or not.

Food Not Food dot app ( 🍔 🚫 🍔 ) Code for building a machine Learning powered app to decide whether a photo is of food or not. See it working live a

Daniel Bourke 48 Dec 28, 2022
Python 3 script for installing kali tools on your linux machine

Python 3 script for installing kali tools on your linux machine

gh0st 2 Apr 20, 2022
the classic version Of torrentleechx #Unmaintained #Archived

TorrentleechX-Classic Old Modified Version Repo #Unmaintained #Archived for support join here working example group Leech Here For Any Issues/Imroveme

XcodersHub 18 Jan 30, 2022
A normal phoneNumber tracker made with python.

A normal phoneNumber tracker made with python.

CLAYZANE 2 Dec 30, 2021
Apache Superset out of box version(Windows 64-bit)

superset_app Apache Superset out of box version (Windows 64bit) prepare job download 3 files python-3.8.10-embed-amd64.zip get-pip.py python_geohash‑0

Steven Lee 9 Oct 02, 2022
mypy plugin for PynamoDB

pynamodb-mypy A plugin for mypy which gives it deeper understanding of PynamoDB (beyond what's possible through type stubs). Usage Add it to the plugi

1 Oct 21, 2022
Official repository for the BPF Performance Tools book

BPF Performance Tools This is the official repository of BPF (eBPF) tools from the book BPF Performance Tools: Linux and Application Observability. Th

Brendan Gregg 1.2k Dec 28, 2022
A good Tool to comment on xmw

A good Tool to comment on xmw

1 Feb 10, 2022