06.Problem Setting & Judging

Problem Setting & Judging – Iconcoderz-2k26

Guide for creating problems, running contests, and processing results.

---

Problem Creation Process

1. Problem Definition

Template:

# Problem Title

## Problem Statement
Clear, concise description of what the problem asks.

## Constraints
- Time limit: 1 second
- Memory limit: 256 MB
- Number of test cases: 1
- n ≤ 10^5
- ai ≤ 10^9

## Input Format
Description of input structure.

## Output Format
Description of expected output.

## Example Input

5 1 2 3 4 5

## Example Output

15

## Explanation
Brief walkthrough of the example.

2. Test Case Creation

Create at least 5 test cases per problem:

1. Sample Test: Provided to participants (easy warm-up).
2. Edge Cases: Boundary values (n=1, n=10^5, etc.).
3. Strong Cases: Medium difficulty, typical scenarios.
4. Weak Case Detection: Cases that catch common mistakes.
5. Stress Test: Large inputs (near constraints).

Store locally:

problems/
├── problem-001/
│   ├── statement.md
│   ├── input1.txt (sample)
│   ├── output1.txt
│   ├── input2.txt (hidden)
│   ├── output2.txt
│   └── editorial.md
├── problem-002/
│   └── ...

3. Problem Difficulty & Distribution

For Iconcoderz-2k26 (approx. 5–6 problems):

Problem	Difficulty	Topics	Expected Solve%
1	Easy	Arrays/Strings	80–90%
2	Easy	Math/Number Theory	60–70%
3	Medium	Sorting/Searching	40–50%
4	Medium	Dynamic Programming	20–30%
5	Hard	Graphs/Trees	5–15%
6 (Optional)	Hard	Advanced DP/Geometry	1–5%

---

HackerRank Contest Setup

1. Create Contest

1. Log into HackerRank → Create → Contests.
2. Contest type: Algorithm (or Custom).
3. Set metadata:
   • Title: Iconcoderz-2k26
   • Description: (brief overview)
   • Start time: 2026-02-23 13:30 UTC+5:30
   • Duration: 4 hours
   • Public/Private: Private (only registered participants)

2. Add Problems

For each problem:

1. Click Add Challenge.
2. Upload problem statement (in Markdown or HTML).
3. Upload sample test cases (visible to participants).
4. Upload hidden test cases (for judging).
5. Set:
   • Time limit: 1–2 seconds (adjust per problem).
   • Memory limit: 256 MB.
   • Score: Typically equal weight per problem or vary by difficulty.

3. Configure Scoring

• ICPC-style: Fewer wrong submissions = higher rank.
• Codeforces-style: Time taken penalty (e.g., 1 min per WA).
• Simple: Points = problems solved (then by time).

For Iconcoderz-2k26, recommend: ICPC-style (problems solved, then penalty time).

4. Enable Languages

Enable: C, C++, Java, Python (minimum).

Optionally: JavaScript, Go, Rust.

5. Create Test

Walkthrough / Tutorial (optional): Upload editorials for 2–3 easy problems so participants see solutions post-contest.

---

During Contest

Monitor

• Real-time submissions (should see activity within first 30 min).
• Check for errors: TLE, MLE, CE, RE.
• Monitor server load (should be fine with ~150 participants).

Common Issues

Issue	Resolution
Participant complains problem is wrong	Review their submission; if logic is correct, clarify statement via contest announcement.
Many RTE errors on a single problem	Check if problem has I/O issues (e.g., requires flushing output).
Submission queue slow	Contact HackerRank support; usually resolves in <5 min.

---

After Contest

1. Export Results

1. HackerRank → Contest → Leaderboard.
2. Click Download leaderboard (CSV).
3. Format: rank, username, email, problems_solved, penalty_time, last_submit_time.

2. Verify Results

• Check top 10 for any anomalies.
• Spot-check a few submissions for correct scoring.

3. Publish Leaderboard

1. Save CSV as leaderboard-2026-02-23.csv.
2. Upload to website (admin panel).
3. Website shows top 10 + option to view full rankings.

4. Post Solutions

Optional (if you want participants to learn):

1. Create solution walkthroughs for 2–3 easiest problems.
2. Post on website or share via email.

---

Future: Custom Judge

If Iconcoderz-2k27 uses a custom judge (instead of HackerRank):

Architecture

Participant submits code
         ↓
   Backend queues submission (SQS/Redis)
         ↓
  Judge worker (Docker sandbox)
  - Compile code
  - Run against each test case
  - Measure time/memory
         ↓
 Write result (AC/WA/TLE) back to DB
         ↓
Frontend updates leaderboard in real-time

Tech Stack

• Queue: AWS SQS or Redis.
• Judge: Custom worker in Docker.
• Scoring: Simple AC/WA, or partial scoring logic.
• Leaderboard: WebSocket or polling.

See Retrospective & Roadmap for more details.

---

Last updated: Jan 2026