MarkItDown: Python tool for converting files and office documents to Markdown
2 by Handy-Man | 0 comments on Hacker News.
Friday, December 13, 2024
New top story on Hacker News: Show HN: I made the slowest, most expensive GPT
Show HN: I made the slowest, most expensive GPT
23 by wluk | 13 comments on Hacker News.
This is another one of my automate-my-life projects - I'm constantly asking the same question to different AIs since there's always the hope of getting a better answer somewhere else. Maybe ChatGPT's answer is too short, so I ask Perplexity. But I realize that's hallucinated, so I try Gemini. That answer sounds right, but I cross-reference with Claude just to make sure. This doesn't really apply to math/coding (where o1 or Gemini can probably one-shot an excellent response), but more to online search, where information is more fluid and there's no "right" search engine + text restructuring + model combination every time. Even o1 doesn't have online search, so it's obviously a hard problem to solve. An example is something like "best ski resorts in the US", which will get a different response from every GPT, but most of their rankings won't reflect actual skiers' consensus - say, on Reddit https://ift.tt/jm8DBXF... - because there's so many opinions floating around, a one-shot RAG search + LLM isn't going to have enough context to find how everyone thinks. And obviously, offline GPTs like o1 and Sonnet/Haiku aren't going to have the latest updates if a resort closes for example. So I’ve spent the last few months experimenting with a new project that's basically the most expensive GPT I’ll ever run. It runs search queries through ChatGPT, Claude, Grok, Perplexity, Gemini, etc., then aggregates the responses. For added financial tragedy, in-between it also uses multiple embedding models and performs iterative RAG searches through different search engines. This all functions as sort of like one giant AI brain. So I pay for every search, then every embedding, then every intermediary LLM input/output, then the final LLM input/output. On average it costs about 10 to 30 cents per search. It's also extremely slow. https://ithy.com I know that sounds absurdly overkill, but that’s kind of the point. The goal is to get the most accurate and comprehensive answer possible, because it's been vetted by a bunch of different AIs, each sourcing from different buckets of websites. Context limits today are just large enough that this type of search and cross-model iteration is possible, where we can determine the "overlap" between a diverse set of text to determine some sort of consensus. The idea is to get online answers that aren't attainable from any single AI. If you end up trying this out, I'd recommend comparing Ithy's output against the other GPTs to see the difference. It's going to cost me a fortune to run this project (I'll probably keep it online for a month or two), but I see it as an exploration of what’s possible with today’s model APIs, rather than something that’s immediately practical. Think of it as an online o1 (without the $200/month price tag, though I'm offering a $29/month Pro plan to help subsidize). If nothing else, it’s a fun (and pricey) thought experiment.
23 by wluk | 13 comments on Hacker News.
This is another one of my automate-my-life projects - I'm constantly asking the same question to different AIs since there's always the hope of getting a better answer somewhere else. Maybe ChatGPT's answer is too short, so I ask Perplexity. But I realize that's hallucinated, so I try Gemini. That answer sounds right, but I cross-reference with Claude just to make sure. This doesn't really apply to math/coding (where o1 or Gemini can probably one-shot an excellent response), but more to online search, where information is more fluid and there's no "right" search engine + text restructuring + model combination every time. Even o1 doesn't have online search, so it's obviously a hard problem to solve. An example is something like "best ski resorts in the US", which will get a different response from every GPT, but most of their rankings won't reflect actual skiers' consensus - say, on Reddit https://ift.tt/jm8DBXF... - because there's so many opinions floating around, a one-shot RAG search + LLM isn't going to have enough context to find how everyone thinks. And obviously, offline GPTs like o1 and Sonnet/Haiku aren't going to have the latest updates if a resort closes for example. So I’ve spent the last few months experimenting with a new project that's basically the most expensive GPT I’ll ever run. It runs search queries through ChatGPT, Claude, Grok, Perplexity, Gemini, etc., then aggregates the responses. For added financial tragedy, in-between it also uses multiple embedding models and performs iterative RAG searches through different search engines. This all functions as sort of like one giant AI brain. So I pay for every search, then every embedding, then every intermediary LLM input/output, then the final LLM input/output. On average it costs about 10 to 30 cents per search. It's also extremely slow. https://ithy.com I know that sounds absurdly overkill, but that’s kind of the point. The goal is to get the most accurate and comprehensive answer possible, because it's been vetted by a bunch of different AIs, each sourcing from different buckets of websites. Context limits today are just large enough that this type of search and cross-model iteration is possible, where we can determine the "overlap" between a diverse set of text to determine some sort of consensus. The idea is to get online answers that aren't attainable from any single AI. If you end up trying this out, I'd recommend comparing Ithy's output against the other GPTs to see the difference. It's going to cost me a fortune to run this project (I'll probably keep it online for a month or two), but I see it as an exploration of what’s possible with today’s model APIs, rather than something that’s immediately practical. Think of it as an online o1 (without the $200/month price tag, though I'm offering a $29/month Pro plan to help subsidize). If nothing else, it’s a fun (and pricey) thought experiment.
Thursday, December 12, 2024
New top story on Hacker News: Show HN: Gentrace – connect to your LLM app code and run/eval it from a UI
Show HN: Gentrace – connect to your LLM app code and run/eval it from a UI
7 by dsaffy | 0 comments on Hacker News.
Hey HN - Doug from Gentrace here. We originally launched via Show HN in August of 2023 as evaluation and observability for generative AI: https://ift.tt/wuPqS7Q Since then, everyone from the model providers to LLM ops companies built a prompt playground. We had one too, until we realized this was totally the wrong approach: - It's not connected to your application code - They don't support all models - You have to rebuild evals for just this one prompt (can't use your end-to-end evals) In other words, it was a ton of work and time to use these to actually make your app better. So, we built a new experience and are relaunching around this idea: Gentrace is a collaborative LLM app testing and experimentation platform that brings together engineers, PMs, subject matter experts, and more to run and test your actual end-to-end app. To do this, use our SDK to: - connect your app to Gentrace as a live runner over websocket (local) / via webhook (staging, prod) - wrap your parameters (eg prompt, model, top-k) so they become tunable knobs in the front end - edit the parameters and then run / evaluate the actual app code with datasets and evals in Gentrace We think it's great for tuning retrieval systems, upgrading models, and iterating on prompts. It's free to trial. Would love to hear your feedback / what you think!
7 by dsaffy | 0 comments on Hacker News.
Hey HN - Doug from Gentrace here. We originally launched via Show HN in August of 2023 as evaluation and observability for generative AI: https://ift.tt/wuPqS7Q Since then, everyone from the model providers to LLM ops companies built a prompt playground. We had one too, until we realized this was totally the wrong approach: - It's not connected to your application code - They don't support all models - You have to rebuild evals for just this one prompt (can't use your end-to-end evals) In other words, it was a ton of work and time to use these to actually make your app better. So, we built a new experience and are relaunching around this idea: Gentrace is a collaborative LLM app testing and experimentation platform that brings together engineers, PMs, subject matter experts, and more to run and test your actual end-to-end app. To do this, use our SDK to: - connect your app to Gentrace as a live runner over websocket (local) / via webhook (staging, prod) - wrap your parameters (eg prompt, model, top-k) so they become tunable knobs in the front end - edit the parameters and then run / evaluate the actual app code with datasets and evals in Gentrace We think it's great for tuning retrieval systems, upgrading models, and iterating on prompts. It's free to trial. Would love to hear your feedback / what you think!
Wednesday, December 11, 2024
Tuesday, December 10, 2024
New top story on Hacker News: Ask HN: Those making $500/month on side projects in 2024 – Show and tell
Ask HN: Those making $500/month on side projects in 2024 – Show and tell
87 by cvbox | 72 comments on Hacker News.
It's the time of the year again, so I'd be interested hear what new (and old) ideas have come up. Previously asked on: 2023 → https://ift.tt/AVPiuM7 2022 → https://ift.tt/EG1Wk09 2021 → https://ift.tt/MR4ym5Z 2020 → https://ift.tt/9hLDFBO 2019 → https://ift.tt/u0mQAlr 2018 → https://ift.tt/wIWJPAH 2017 → https://ift.tt/r1VXNdu
87 by cvbox | 72 comments on Hacker News.
It's the time of the year again, so I'd be interested hear what new (and old) ideas have come up. Previously asked on: 2023 → https://ift.tt/AVPiuM7 2022 → https://ift.tt/EG1Wk09 2021 → https://ift.tt/MR4ym5Z 2020 → https://ift.tt/9hLDFBO 2019 → https://ift.tt/u0mQAlr 2018 → https://ift.tt/wIWJPAH 2017 → https://ift.tt/r1VXNdu