If I knew a way to make Augment even more efficient, would you be interested? I think that I have developed a revolutionary prompt engineering framework that creates a software stack in-context and has a similar effect of fine tuning the model without fine tuning the model.
It leverages the concepts from my research about reinforcement learning by creating a pseudo order of operations (a literal one but not in the same sense as a program) that uses mermaid diagrams to define a "workflow" where the nodes are named for "workflows" that I have defined with xml.
The mermaid "workflow" get's called by "event handlers" that I also defined in xml and the so `xml >> diagram >> xml/workflow with steps >> xml/functions with prompts to develop a plan or execute a task >> xml/evaluation >>
* creator[created task solution]
* critic[Plan could be better]
* defender[I think the plan is good but here are additional thought
* judge[final solution]
Accuracy, Efficiency, Process, Innovation.
You can read more about my approach here:
github.com/entrepeneur4lyf/c…
I have created 2 "products" that work hand-in-hand. The primary is what I refer to as "Engineered Meta-Cognitive Workflow Architecture" which leverages the workflow I described above but is based on the foundational ideas in the Cognitive Prompt Architecture you can find in the repo above.
These are 6 distinct of what I call Thought Domain Prompt Frameworks (and there are almost 20 in total that I am working to develop into digestible material).
Essentially, these "thought domains" attempt to capture the most efficient way to think about the solutions to different problem categories and define scorable criteria by which the LLM can gauge it's performance.
This technique, combined with a 3 layer project specific memory bank system developed by Nick Baumann from
@cline (who discovered that the models would FOLLOW A MERMAID DIAGRAM WORKFLOW!!! Just amazing, tbh) and my own experimentation with using xml tags in prompts as advised by
@AnthropicAI led to some great results. Less hallucinations but not perfect.
The real breakthrough came while I was creating a codebase indexing and real-time context feature for my own purposes (which
@dani_avila7 graciously allowed to be published on his repo about knowledge graphs) and research Monte Carlo Tree Search with UCT to score results to produce the most relevant context based on multiple criteria.
It was a crazy realization while I was sitting in a Hobby Lobby parking lot... the answer was self scoring based on defined criteria that clicked when I was reading a
@deepseek_ai paper about training with GRPO (Process Supervision RL with GRPO
arxiv.org/pdf/2402.03300). It wasn't that I planned to implement GRPO, specifically. It was the research into MCTS and Reward/Penalty (this was used to train Alphazero to win at games) and their references to the reward that made it click. I immediately developed the same exact criteria I am still using in the current architecture on my phone, prompted Claude 3.5 Sonnet and watched as it's output literally seemed to speed up. It seemed excited to meet the challenge. I know that's crazy... but let me tell you. They COMPLY.
While not an implementation of GRPO or an actual implementation any of these algorithms or paradigms, of In this case, I have developed a multi-layered approach to doing so. Both with scoring the proposed solution to a problem and the result.
The result of this discovery has been more than 99% efficiency over an 8 hour period of coding the same project. Very little hallucinations, laziness, improper coding standards, etc. I have the logs to prove it because the framework requires they log every task.
You can read more about the coding product here along with screenshots and guides for best practice.
entrepeneur4lyf.github.io/en…
The prompt is, in total, between 6000-6200 tokens but I have broken it into 2 parts that seem to function fine... not as good, tbh, as when I could give it all to them in a cohesive prompt but it works.
You will find 2 files in this gist. One is `.augmentrules` that contains the xml and mermaid diagram parts of the prompt. The other is `Meta-Cognitive-Augment` that you will copy and paste into "Augment > Chat: User Guidelines" in vscode settings (not Augment settings).
Since it sounds like you are working on an existing codebase. Follow this process.
Run this command in the src or app directory (I am not familiar with Svelte).
`tree >
structure.md` <-- you may have to remove http if you copy and paste
Then edit the file and make sure it only includes the files you want to have the model look at. I remove all the extra info like file count, etc. Attach the structure file to the chat and give them this prompt.
Prompt: “Read .augmentrules in the project root and execute the sessionStart Event Handler from the Meta Cognitive Workflow. When initialization is completed, do a thorough code review of every file in the project structure I have provided +1 for each file, document current state of each file +1, identify any implementation issues +1, missing dependencies +1, missing functionality +1, missing doc comments +1 for each file. Do not skip any files or assume their contents. Calculate the total score for each file before you begin and evaluate your performance based on that baseline score. The provided criteria are simple tasks and I will penalize you for any missed criteria or substandard score. This is a simple directive and only a perfect score will be acceptable for this directive. Log your analysis for each file noting file name, date and time and total possible score in your task log. Be thorough - provide analysis log for each criteria in order and analysis summary when criteria are complete for each file. Note completion time at the bottom and proceed to the next file. Create a memory detailing this directive and note the path to `
structure.md`. Create the task log before you begin and update your memory bank and task log after each file has been analyzed. We will use this analysis to develop a projectBrief once this directive is complete. Proceed to the task.”
Typically, the model will just go right to creating the projectBrief. If not, See "Troubleshooting" below.
Once it's done with the codebase review, which will be detailed... tell it to use what it has learned and create an implementation plan.
For the absolute best results - because it is the CORRECT way to develop software. Scroll paste the next paragraph.
***
If it doesn't create an implementation plan and take off creating tasklogs, tell it to "Run `TaskStart` Workflow Event Handler and use the PRD to create an implementation plan" If it isn't cooking after that, your PRD might be insufficient. It will probably tell you that though. *** Follow best practices. If you notice the model not being compliant, it's time to start a new session/chat. "Just click the "+" icon, do the "sessionStart Event Handler from the Meta Cognitive Workflow" bit from the start of this book I just typed out. It will catch up. Continue your project. If it isn't writing task-logs after every task - same thing... sessionStart Event Handler.
I know this is super long and I apologize... but it is worth it because it will change your entire experience, I promise you.
***
SCROLL HERE -
Optionally, once the model completes the codebase review, you can give it the model-instructions, project-overview and the templates folder and watch it create a complete set of documentation using this workflow
github.com/entrepeneur4lyf/e…
I am currently developing a tool to do all of this from an MCP server. If you are interested, give me a follow.
Feel free to DM me if you need any assistance.