Doing Machine Learning Research: A Guide for Your Thesis

For many students, the final-year project or thesis is the first real research experience: an open question, no answer key, and months to find something new. It can be the most rewarding part of your degree — or the most stressful. This closing lecture distils advice I give to the students I supervise, so that your research is rigorous, ethical and genuinely useful.

Step 1: choose a good question#

A good research question is:

• Specific: "Does continued pretraining on 50 MB of Bangla news improve NER F1 over XLM-R on dataset X?" rather than "Improve Bangla NLP".
• Feasible: achievable with your data, compute and time (be honest about GPU access).
• Relevant: someone cares about the answer — a community, an organisation, other researchers.
• Testable: you can design an experiment whose outcome could prove you wrong.

Good sources of questions: limitations sections of recent papers, failures you observe when applying existing methods to your language or domain, needs of local organisations, and replication of important results under new conditions. Replication and careful evaluation studies are legitimate, valuable research, especially for low-resource languages and new contexts.

Step 2: review the literature efficiently#

• Start from 2–3 surveys and follow citations backwards and forwards (Semantic Scholar's citation graph helps).
• Build a table: paper, method, data, metric, results, limitations.
• Identify the gap your work addresses, and the strongest existing baselines.
• Do not wait until you have read everything — alternate reading with early experiments.

Step 3: design rigorous experiments#

1. Define metrics and evaluation data before running experiments. Keep a test set untouched until the end.
2. Start with strong, simple baselines (e.g. TF-IDF + logistic regression, a fine-tuned small transformer) and reproduce published numbers where possible.
3. Change one thing at a time; plan ablations from the start.
4. Multiple seeds and confidence intervals; statistical tests for key comparisons.
5. Fair comparisons: equal tuning budgets for baselines and your method.
6. Error analysis: qualitative examination of failures often yields the most interesting insights.
7. Track everything (experiment tracking, versioned data and code) so you can reproduce every number in your thesis.

Experiment plan template
Question:            Does X improve Y on Z?
Hypothesis:          X improves macro-F1 by >= 2 points over baseline B.
Data & splits:       dataset D (v1.2), fixed train/val/test, test used once.
Baselines:           B1 (simple), B2 (strong, published), each tuned with 20 trials.
Metrics:             macro-F1 (primary), per-class F1, calibration (secondary).
Seeds:               5 per configuration; report mean ± std and 95% CI.
Ablations:           remove component a; remove component b.
Compute budget:      ~40 GPU-hours.
Risks & fallback:    if D too small, use cross-validation; if no GPU, use smaller models.

Step 4: avoid common pitfalls#

Step 5: write clearly#

Structure: abstract, introduction (problem, gap, contributions), background and related work, method, experimental setup, results, analysis/discussion, limitations, conclusion.

Writing advice:

• Write the contribution list early and let it guide everything.
• One idea per paragraph; topic sentence first.
• Every figure and table must be referenced and explained in the text; captions should be self-contained.
• Define notation once and use it consistently.
• Report numbers with appropriate precision and uncertainty.
• Be honest about limitations — reviewers and examiners respect it.
• Revise: the first draft is for you; later drafts are for readers. Ask peers to read it.

Step 6: research ethics#

• Data ethics: use data you are permitted to use; obtain ethics approval when working with human participants or personal data; anonymise and secure data; respect licences.
• Community involvement: when research concerns a community (e.g. a language community or displaced population), involve its members as partners, not just subjects; share results back.
• Honesty: never fabricate or selectively report results; disclose AI assistance in writing according to your institution's policy; cite properly — plagiarism includes paraphrasing without attribution.
• Impact: consider potential misuse and harms; include an impact statement.
• Credit: acknowledge everyone who contributed, including annotators.

Step 7: share your work#

Release code and (where permitted) data with documentation; write a blog post; present at local meetups, student conferences and workshops (many major conferences host workshops for specific languages, regions and application areas, often welcoming student work). Shared, reproducible work builds your reputation and helps others build on it.