JSONDecodeError while running pyspark and flask_ngrok on google colab - flask

Question

I am running pyspark to retrieve a table called books_df from S3 and convert it to a spark dataframe to use it further in my application.
Pyspark Code:
import pyspark
from pyspark import SparkContext, SparkConf
from pyspark.sql import SparkSession
os.environ["JAVA_HOME"] = "/usr/lib/jvm/java-11-openjdk-amd64"
os.environ["SPARK_HOME"] = "/content/drive/<<path to curr dir>>/spark-3.1.2-bin-hadoop2.7"
findspark.init()
conf = pyspark.SparkConf().setAppName('appName').setMaster('local')
sc = pyspark.SparkContext(conf=conf)
spark = SparkSession(sc)
spark.conf.set("spark.sql.execution.arrow.enabled", "true")
df = spark.createDataFrame(books_df)
df.show(3)
+--------------------+--------------------+--------------------+-----------+
| book_authors| book_desc| book_edition|book_format|
+--------------------+--------------------+--------------------+-----------+
| Suzanne Collins|Winning will make...| null| Hardcover|
|J.K. Rowling|Mary...|There is a door a...| US Edition| Paperback|
| Harper Lee|The unforgettable...| 50th Anniversary| Paperback|
+--------------------+--------------------+--------------------+-----------+
only showing top 3 rows
flask_ngrok section:
from flask import Flask, render_template,redirect, url_for, request, session, abort, flash
from flask_ngrok import run_with_ngrok
app = Flask(__name__, template_folder='/content/drive/<<path to curr dir>>/templates',static_folder='/content/drive/<<path to curr dir>>/static')
run_with_ngrok(app)
#app.route('/')
def upload():
return(render_template('upload.html'))
if __name__ == '__main__':
app.run()
When running this, I get the following error:
* Serving Flask app "__main__" (lazy loading)
* Environment: production
WARNING: This is a development server. Do not use it in a production deployment.
Use a production WSGI server instead.
* Debug mode: off
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Exception in thread Thread-25:
Traceback (most recent call last):
File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner
self.run()
File "/usr/lib/python3.7/threading.py", line 1177, in run
self.function(*self.args, **self.kwargs)
File "/usr/local/lib/python3.7/dist-packages/flask_ngrok.py", line 70, in start_ngrok
ngrok_address = _run_ngrok()
File "/usr/local/lib/python3.7/dist-packages/flask_ngrok.py", line 36, in _run_ngrok
j = json.loads(tunnel_url)
File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Important note: When I add sc.stop() after the pyspark section, flask_ngrok works perfectly fine but I cannot access the pyspark dataframe df as the sparkContext is closed.
Is pyspark and flask_ngrok trying to access the same port and thus, giving the error?