Automating My GitHub Profile: From Manual Commits to a README Generator

Introduction

That summer after the gaokao, China’s university entrance exam, I skipped the celebrations and plunged into the world of code. After two months of exploring algorithms and data structures, I wanted to record the experience and give that effort a visible trace. To do this, I turned to GitHub. My plan was to build a personal homepage that was uniquely mine and could dynamically show the course of my learning.

I learned that a regular git push would timestamp all the commits with the current time, leaving the GitHub contribution graph unable to faithfully and precisely reconstruct my learning over the summer.

To solve this problem, and to create a traceable, visual record of my future learning, I designed and carried out this manual migration and reconstruction of the version history.

This post records the entire process and what went through my mind along the way, from struggling to type Git commands correctly to finally automating the work with Python scripts.

Doing It Manually: The Basic Workflow and Troubleshooting

I decided to use git commit to create a commit with a historical timestamp for each solution file. This approach was workable, but while carrying it out manually, I encountered and resolved a series of typical command-line and Git configuration issues. Here is a review of them.

The Core Approach

The technical foundation was the --date option provided by Git’s commit command. It lets users specify a custom timestamp when creating a commit, making it possible to build a version history retrospectively.

The basic workflow I designed was as follows:

1. Add the code file to the working directory.

2. Manually add a standardized comment header in the first four lines of the file, in this format:

   // Problem: 练习平台 题号 题目
// Link: 题目链接
// Author: nine19een
// Date: 日期
//
// 代码正文

   The Chinese placeholders mean, in order: practice platform, problem ID and title; problem URL; date; and the code itself.

3. Use git add to put the file changes in the staging area.

4. Run git commit with the specific historical date to create the commit.

An example of the core command:

git commit --date="YYYY-MM-DD HH:MM:SS" -m "Creat 文件名"

Here, --date specifies the historical timestamp. I could get it from the submission history on the practice platform and manually substitute it into the command.

Configuration and Troubleshooting

While putting this workflow into practice, I encountered and resolved the following typical configuration and command-line issues.

Case 1: Navigating to the Wrong Path — No such file or directory

• What happened:
  In Git Bash, using cd with a standard path copied from Windows File Explorer failed to change directories and returned No such file or directory.

• Analysis:
  This error came from the different path notations used by Windows and Unix-like environments such as Git Bash. There were two main points:

  1. Path separators: Windows uses \, whereas Git Bash needs /. So copying a path directly from Windows File Explorer and pasting it into Git Bash does not work!
  2. A complete path: The path structure must be complete and correct, starting from the drive letter.

• Solution:

  1. Fix the format: If copying and pasting the path, manually replace every \ with / and make sure the path is complete, for example, C:/Users/YourName/Documents/....

  2. Use the GUI to help: A more efficient, less error-prone approach is to type cd and then drag the target folder from File Explorer straight into Git Bash. This automatically produces a complete path in the correct syntax. I strongly recommend this approach; it is harder to get wrong.

     

     Dragging the file into Git Bash automatically generates its full path.

     

     Then just press Enter.

Case 2: Being Outside the Repository — fatal: not a git repository

• What happened:
  After successfully running git clone, I ran commands such as git status or git add in the current directory—the directory where I had run clone. Git returned a fatal error saying that the current directory was not a Git repository.

• Analysis:
  git clone creates a new subdirectory in the current directory as the repository root. All Git operations must run inside this new directory or one of its descendants, because the root contains the required .git metadata folder. When the error occurred, I was in the repository’s parent directory.

• Solution:
  After git clone, use cd <repository-name> to enter the root of the newly cloned repository before carrying out any further Git operations.

Case 3: A Failed Network Connection — schannel: failed to receive handshake, SSL/TLS connection failed

• What happened:
  After sorting out the local paths and repository location, I ran into the most difficult blocker. Any network operation that needed to communicate with the remote server, such as git push, git pull, or git fetch, would fail after a long wait and return this low-level network error: schannel: failed to receive handshake, SSL/TLS connection failed.

  The most confusing part was that, at the same time, my browser could open github.com quickly and without any trouble.

• Analysis:
  “It works in the browser but not on the command line” is a very typical symptom of a proxy configuration issue.

  The root of the problem was that I was running a proxy tool such as Clash on my computer to improve network access. My browser was correctly configured to access GitHub through the proxy, so its connection was smooth. Git Bash, however, is a separate command-line environment and does not automatically use the system proxy settings by default.

  As a result, Git was still trying to connect directly to GitHub’s servers. My network environment interfered with that direct route, causing a timeout during the SSL/TLS handshake. schannel is the name of the built-in Windows security library that handles this process.

• Solution:
  The key was to manually configure the Git command-line tool to use the same proxy server as the system.

  This took two steps:

  1. Find the proxy port:
     First, find the local port on which Clash listens for HTTP/HTTPS traffic. Here I use my own setup, Clash Verge, as an example.

     

     Screenshot guide: “设置” means Settings, and “端口设置” means Port Settings. The circled port in this example is 7897.

  2. Configure Git’s global proxy:
     Next, run the following two commands in Git Bash, pointing both http.proxy and https.proxy at the local proxy’s listening address (http://127.0.0.1:端口号). P.S. 127.0.0.1 is a special IP address that always refers to your own machine. In these commands, 端口号 is the port-number placeholder.

     • Point Git’s HTTP traffic at the local port:

       git config --global http.proxy http://127.0.0.1:端口号

     • Point Git’s HTTPS traffic at the local port too (the crucial part):

       git config --global https.proxy http://127.0.0.1:端口号

After running these two commands and successfully “turning on the proxy” for Git, I ran git fetch again, and the connection problem was resolved.

Case 4: A Rejected Push — ! [rejected] main -> main (fetch first)

• What happened:
  After making a local commit, I ran git push, but the remote server rejected it. The message said that the remote repository contained changes that were missing locally.

• Analysis:
  This is a Git safeguard intended to prevent a local push from accidentally overwriting commits made by someone else—or by me elsewhere—on the remote repository. It commonly happens when a new commit has been made remotely since the local clone or most recent pull, for example, editing README.md directly on the GitHub website.

• Solution:
  Always run git pull before every git push, so the local and remote repository files stay synchronized.

  1. Run git pull to download the latest remote changes and merge them automatically.
  2. Resolve any merge conflicts that arise (for a personal project, the automatic merge will usually succeed), then run git push again.

Case 5: A Line-Ending Warning — LF will be replaced by CRLF

• What happened:
  When using git add on a file, Git Bash displayed a warning that the file’s LF line endings would be replaced with CRLF.

• Analysis:
  This happens because operating systems use different line-ending conventions. Unix/Linux/macOS use LF, while Windows uses CRLF. Git’s built-in core.autocrlf feature can automatically convert between them to keep line endings consistent in the repository. The warning is a sign that this feature is at work.

• Solution:
  Ignore the risk and continue (heh). No action is needed; the warning can safely be ignored. This is normal Git behavior, meaning that it is handling cross-platform compatibility for us in the background.

Case 6: Managing the Staging Area — Undoing an Accidental git add

• What happened:
  After running git add ., I noticed that I had staged some unnecessary files or incorrect changes and needed to undo that before making a commit.

• Analysis:
  This is a very common need in a Git workflow: a command that safely moves files out of the staging area and back to the working directory without losing any code changes.

• Solution:

  1. Unstage everything: git reset clears the entire staging area in one go.

     git reset

  2. Unstage one file: git restore --staged <file> removes a specific file from the staging area.

     git restore --staged <filename>

  Neither approach changes the contents of files in the working directory, so both are very safe.

Automating It: From Manual Labor to Building a System

After four hours of manually migrating historical commits, I immediately realized that creating and maintaining a table for these hundred-plus solution files in README.md—with a consistent format, accurate information, and date ordering—would be extremely time-consuming and error-prone. be the death of me.

So I decided to write Python scripts to automate the entire README.md update process. P.S. Laziness is humanity’s greatest productive force.

The Core Approach: Two Stages

To reduce complexity and keep each step stable and reliable, I split the automation task into two separate but sequentially related projects:

1. Project One: Standardizing Source-Code Comments

   • Goal: Traverse all the source files and update any whose comments do not follow the standard. For the format, see Doing It Manually: The Basic Workflow and Troubleshooting → The Core Approach → Step 2.

   • Purpose: Provide the later README generator with a consistent, reliable data source that can be parsed offline.

2. Project Two: A README Generator

   • Goal: Read all the standardized source files, extract the metadata from their header comments, and generate the complete final Markdown table containing every problem and solution for README.md.

   • Purpose: Completely replace the work of manually editing the README.

Project One: The Source-Comment Standardization Script

Case 1: Inconsistent Encodings — Garbled Chinese Text

• What happened:
  When my script tried to read .cpp files, or when I opened files cloned from GitHub in CLion, their Chinese comments appeared as garbled text.

• Analysis:
  This was a typical character-encoding mismatch. GitHub and CLion use UTF-8 by default, which supports languages from around the world. The Dev-C++ I had previously used for practice on Windows, however, saved files in the local GBK encoding by default. When a program tries to read a GBK file as UTF-8, the text becomes garbled.

• Solution:
  Enforce UTF-8 throughout the development chain.

  1. Migrate the development environment: Stop using the old IDE with poor UTF-8 support and move my main development work to the modern, UTF-8-by-default CLion.

  2. Specify encodings in the script: Explicitly set the encoding argument whenever using Python’s open(). Use encoding='utf-8-sig' for reading to support UTF-8 files with a BOM on Windows, and encoding='utf-8' for writing.

     1 2 3 4 5 6 7

     # 读取时指定编码 with open(file_path, 'r', encoding='utf-8-sig') as f: lines = f.readlines() # 写入时同样指定编码 with open(file_path, 'w', encoding='utf-8') as f: f.writelines(lines)

The Final Script: update_sources.py

The script’s core logic is to traverse every C++ file in the specified directory and check whether it already has a standard header. If it does not, the script parses the problem ID from its filename, requests the problem URL to retrieve the full title, and gets the date of the file’s first commit from Git history. It then combines this metadata into a standard comment block and writes it back to the start of the file.

Once I had explained my requirements thoroughly to Gemini, a Python script for automatically updating the comments was born.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

import os
import requests
from bs4 import BeautifulSoup
import re
import time

REPO_FOLDER = "Coding-Practice"

def get_luogu_title(url):
    try:
        headers = {
            'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'
        }
        response = requests.get(url, headers=headers, timeout=10)
        response.raise_for_status()

        soup = BeautifulSoup(response.text, 'lxml')

        title_tag = soup.find('h1')
        if title_tag:
            return title_tag.get_text(strip=True)
        else:
            return None
    except requests.exceptions.RequestException as e:
        print(f"    - 网络错误: 无法访问 {url} - {e}")
        return None

def process_file(file_path):
    try:
        with open(file_path, 'r', encoding='utf-8-sig') as f:
            lines = f.readlines()

        if not lines:
            print(f"  - 跳过 (文件为空): {os.path.basename(file_path)}")
            return

        first_line_content = lines[0].strip()
        if first_line_content == "// Problem:  Luogu":
            print(f"  - 处理中: {os.path.basename(file_path)}")

            link_url = ""
            for line in lines:
                if "// Link:" in line:
                    match = re.search(r'https?://[^\s]+', line)
                    if match:
                        link_url = match.group(0)
                    break

            if not link_url:
                print(f"    - 错误: 在文件中未找到有效的 Link URL。")
                return

            title = get_luogu_title(link_url)
            time.sleep(0.5)

            if title:
                lines[0] = f"// Problem:  Luogu {title}\n"
                with open(file_path, 'w', encoding='utf-8') as f:
                    f.writelines(lines)
                print(f"    - 更新成功！标题已设置为: {title}")
            else:
                print(f"    - 更新失败: 未能从 {link_url} 获取标题。")
        else:
            print(f"  - 跳过 (已处理): {os.path.basename(file_path)}")

    except Exception as e:
        print(f"  - 处理文件时发生未知错误: {os.path.basename(file_path)} - {e}")

def main():
    if not os.path.isdir(REPO_FOLDER):
        print(f"错误: 文件夹 '{REPO_FOLDER}' 不存在。请确保脚本和该文件夹在同一目录下。")
        return

    print(f"--- 开始扫描文件夹: {REPO_FOLDER} ---")

    all_files = sorted([f for f in os.listdir(REPO_FOLDER) if f.startswith("Luogu") and f.endswith(".cpp")])

    for filename in all_files:
        full_path = os.path.join(REPO_FOLDER, filename)
        process_file(full_path)

    print("\n--- 扫描完成 ---")

if __name__ == "__main__":
    main()

Project Two: The README Generator Script

Once every source file had standardized metadata, generating the README became a straightforward task.

Case 1: The Scraper Stopped Working

• What happened:
  While developing update_sources.py, the scraper function for retrieving Luogu problem difficulties kept failing despite repeated attempts. It returned N/A or empty page content even though the URLs opened normally in the browser.

• Analysis:
  I wrote a separate script to save the raw HTML returned by requests and found that I was receiving a Cloudflare human-verification page rather than the Luogu problem page. My requests had probably been identified and blocked by Cloudflare’s anti-scraping mechanism.

• Solution:

  1. Iterating on the approach: I tried several scraping strategies: changing the User-Agent, looking for specific HTML tags (Luogu’s difficulty labels use <span>), and even using regular expressions to match data scripts directly in the page. All failed because of the anti-scraping mechanism.
  2. The final approach: It suddenly occurred to me that I could save the problems and their difficulties locally straight from my Luogu profile, turning them into static data.
  

  Difficulty-label guide: “入门” = Beginner; “普及−” = Basic−; “普及/提高−” = Basic/Intermediate−; “普及+/提高” = Basic+/Intermediate. These are the Luogu labels shown in the screenshot.

  I therefore made a key architectural decision: give up dynamic scraping and switch to a static local data source.

  I manually organized the difficulty information for every problem into a Python dictionary and built it directly into the script.

  1
  2
  3
  4
  5
  6
  # 本地难度数据库示例
  DIFFICULTY_DATA = {
      "P1001": "入门",
      "P1002": "普及−",
      # ... and so on
  }
  
  

  This decision freed the script from Cloudflare entirely, making it 100% reliable and extremely fast, as well as easy to maintain in the long term.

Case 2: An Oversight in the Logic

• What happened:
  The first version successfully generated table rows for all the new problems. During later iterations, however, I found that it dropped existing records from platforms other than Luogu, such as Codeforces, in README.md.

• Analysis:
  My script only considered “generating the new,” without considering “keeping the old” or “merging and sorting.”

• Solution:
  Refactor the core logic to support multiple platforms:

  1. Read everything: When reading the old README, load every table row into memory rather than filtering for Luogu problems alone.
  2. Process incrementally: Generate new table rows only for files whose filenames do not appear in the existing records.
  3. Merge and sort: Combine the new rows and all the existing rows into a single list, then sort the whole list by date in descending order.

The Final Script: generate_readme.py

This is the final version after multiple refactorings and bug fixes. It combines the local difficulty database, support for multiple platforms, incremental updates, and sorting of the full dataset.

Running it produces an update.txt file containing the final, perfectly sorted table of old and new problems. All I need to do is copy it in full and replace the old table in README.md.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183

import os
import re
from datetime import datetime
from urllib.parse import unquote

# --- Configuration ---
GITHUB_USERNAME = "nine19een"
REPO_NAME = "Coding-Practice"
SOURCE_CODE_PATH = "Coding-Practice"
README_FILE_PATH = "README.md"
OUTPUT_FILENAME = "update.txt"
# ---------------------

# ==============================================================================
# Local Difficulty Database
# ==============================================================================
DIFFICULTY_DATA = {
    "P1001": "入门", "P1046": "入门", "P1047": "入门", "P1085": "入门",
    # 略
}
# ==============================================================================

BADGE_TEMPLATES = {
    "入门": '<img src="https://img.shields.io/badge/入门-FE4C61?style=for-the-badge&textColor=white" alt="入门">',
    "普及−": '<img src="https://img.shields.io/badge/普及−-F39C11?style=for-the-badge&textColor=white" alt="普及−">',
    # 略
}

def load_existing_entries(readme_path):
    existing_entries = []
    in_table_section = False
    if not os.path.exists(readme_path):
        return existing_entries
    try:
        with open(readme_path, 'r', encoding='utf-8') as f:
            for line in f:
                if line.strip().startswith('| :---'):
                    in_table_section = True
                    continue
                if in_table_section and (not line.strip().startswith('|') or not line.strip()):
                    in_table_section = False

                if in_table_section:
                    date_match = re.search(r'\|\s*(\d{4}-\d{2}-\d{2})', line)
                    date_str = date_match.group(1) if date_match else "1970-01-01"
                    existing_entries.append({
                        "date_obj": datetime.strptime(date_str, '%Y-%m-%d'),
                        "row": line.strip()
                    })
    except Exception as e:
        print(f"Warning: Failed to read README file: {e}")
    return existing_entries

def parse_source_file_metadata(file_path):
    metadata = {'platform': 'unknown'}
    try:
        with open(file_path, 'r', encoding='utf-8-sig') as f:
            content = f.read()

        date_match = re.search(r'//\s*Date:\s*(\d{4}-\d{2}-\d{2})', content)
        if date_match: metadata['date'] = date_match.group(1)

        link_match = re.search(r'//\s*Link:\s*(https?://[^\s]+)', content)
        if link_match: metadata['link'] = link_match.group(1)

        problem_match = re.search(r'//\s*Problem:\s*(.*)', content)
        if problem_match:
            line_content = problem_match.group(1).strip()
            if line_content.lower().startswith("luogu"):
                metadata['platform'] = 'luogu'
                metadata['details'] = line_content[5:].strip()
            elif line_content.lower().startswith("codeforces"):
                metadata['platform'] = 'codeforces'
                metadata['details'] = line_content[10:].strip()
            elif line_content.lower().startswith("opj"):
                metadata['platform'] = 'opj'
                metadata['details'] = line_content[3:].strip()
        return metadata
    except Exception as e:
        print(f"    - Error: Failed to parse file {os.path.basename(file_path)}: {e}")
        return None

def build_markdown_row(metadata, filename):
    date = metadata.get('date', 'N/A')
    platform = metadata.get('platform')
    details = metadata.get('details', '')
    link = metadata.get('link', '#')

    problem_md, difficulty_badge = f"[{platform}-{details}]({link})", ""

    if platform == 'luogu':
        pid_match = re.search(r'([PB]\d+)', details)
        if pid_match:
            problem_id = pid_match.group(1)
            title = re.sub(r'「.*?」|\[.*?\]|【.*?】', '', details).replace(problem_id, '').strip()
            problem_md = f"[{'洛谷'}-{problem_id}-{title}]({link})"
            difficulty_text = DIFFICULTY_DATA.get(problem_id, "未定义")
            difficulty_badge = BADGE_TEMPLATES.get(difficulty_text, BADGE_TEMPLATES["未定义"])
    elif platform == 'codeforces':
        id_letter_match = re.search(r'-([A-F]\d?)-', filename) or re.search(r'\s+([A-F]\d?)\.', details)
        if id_letter_match:
            id_letter = id_letter_match.group(1)[0]
            badge_key = f"Div.4 {id_letter}"
            difficulty_badge = BADGE_TEMPLATES.get(badge_key, "**CF**")
        else:
            difficulty_badge = "**CF**"
        problem_md = f"[Codeforces-{details}]({link})"
    elif platform == 'opj':
        pid_match = re.search(r'(\d+)', details)
        problem_id = pid_match.group(1) if pid_match else ''
        title = details.replace(problem_id, '').strip()
        problem_md = f"[opj-{problem_id}-{title}]({link})"
        difficulty_badge = BADGE_TEMPLATES.get("基础", "")

    safe_filename = filename.replace('+', '%2B')
    solution_md = f"[View Code (C++)](https://github.com/{GITHUB_USERNAME}/{REPO_NAME}/blob/main/{safe_filename})"

    return {
        "date_obj": datetime.strptime(date, '%Y-%m-%d'),
        "row": f"| {date} | {problem_md} | {difficulty_badge} | {solution_md} |"
    }

def main():
    print("--- Initializing README Generation ---")

    existing_entries = load_existing_entries(README_FILE_PATH)
    existing_filenames = set(unquote(re.search(r'/main/(.*)\)', entry['row']).group(1)) for entry in existing_entries if re.search(r'/main/(.*)\)', entry['row']))

    print(f"Found {len(existing_entries)} existing entries in README.")

    all_entries = existing_entries

    if not os.path.isdir(SOURCE_CODE_PATH):
        print(f"Error: Source code directory '{SOURCE_CODE_PATH}' not found.")
        return

    print(f"\n--- Scanning for new source files in {SOURCE_CODE_PATH} ---")

    all_cpp_files = sorted([f for f in os.listdir(SOURCE_CODE_PATH) if f.endswith(".cpp")])
    new_files_processed = 0

    for filename in all_cpp_files:
        if filename in existing_filenames:
            continue

        print(f"  - Processing new file: {filename}")
        new_files_processed += 1

        file_path = os.path.join(SOURCE_CODE_PATH, filename)
        metadata = parse_source_file_metadata(file_path)

        if not metadata or 'date' not in metadata:
            print(f"    - Warning: Could not parse required metadata from {filename}. Skipping.")
            continue

        new_entry = build_markdown_row(metadata, filename)
        all_entries.append(new_entry)

    if new_files_processed == 0:
        print("\n--- No new files to add. README is up to date. ---")
        return

    print(f"\n--- Processed {new_files_processed} new files. Generating final table. ---")

    all_entries.sort(key=lambda x: x['date_obj'], reverse=True)

    try:
        with open(OUTPUT_FILENAME, 'w', encoding='utf-8') as f:
            f.write("| Date       | Problem                                                    | Difficulty                                                                                                   | Solution                                                                                  |\n")
            f.write("| :--------- | :--------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------- |\n")
            for entry in all_entries:
                f.write(entry['row'] + '\n')

        print("\n" + "="*50)
        print(f"Success! The complete and sorted table has been saved to:")
        print(f"{os.path.abspath(OUTPUT_FILENAME)}")
        print("Please copy the entire content of this file and replace the old table in your README.")
        print("="*50)
    except Exception as e:
        print(f"\nError! Failed to write output file: {e}")

if __name__ == "__main__":
    main()

The Final Result

After all that manual migration, automation-script building, and README.md generation, my GitHub Profile finally became a dynamically updated, information-rich personal technical record.

It now contains both a contribution graph that faithfully reflects when I studied and a detailed, consistently formatted practice log that I can keep updating with automation scripts.

Screenshot guide: The practice table lists dates, problems, difficulty badges, and C++ solution links. Its Chinese difficulty badges use the same labels explained above.

This Profile will faithfully record every step I take through my four years at university.

Come visit my ➡️ GitHub profile ⬅️ for the latest progress and project source code. Advice from more experienced developers is always welcome~