feat(connectors): add base connector and registry for detection
Idea is to have a "plugin-type" system, where new connectors can extend the `BaseConnector` class and implement the fetch posts method. These are automatically detected by the registry, and automatically used in new Flask endpoints that give a list of possible sources. Allows for an open-ended system where new data scrapers / API consumers can be added dynamically.
This commit is contained in:
@@ -111,6 +111,20 @@ def get_user_datasets():
|
||||
current_user = int(get_jwt_identity())
|
||||
return jsonify(dataset_manager.get_user_datasets(current_user)), 200
|
||||
|
||||
@app.route("/datasets/sources", methods=["GET"])
|
||||
@jwt_required()
|
||||
def get_dataset_sources():
|
||||
return jsonify({""})
|
||||
|
||||
@app.route("/datasets/scrape", methods=["POST"])
|
||||
@jwt_required()
|
||||
def scrape_data():
|
||||
if "sources" not in request.form:
|
||||
return jsonify({"error": "Data source names are required."}), 400
|
||||
|
||||
sources = request.form.get("sources")
|
||||
|
||||
|
||||
@app.route("/datasets/upload", methods=["POST"])
|
||||
@jwt_required()
|
||||
def upload_data():
|
||||
|
||||
Reference in New Issue
Block a user