Tested The Model.

#1
by dadadies - opened

Here are the results and what I found. Not a critique but just an observation. The default generations are superficially fancy or whatever you want to call it. They often dont make sense or if anything distract from the story. Each time I ask to provide a simple version and the story makes more sense without really losing any detail. I get the fancy so car poetic style but it seems too fancy. I can only imagine how it would write about a person drinking a cup of water. At the very least it doesn't start like the typical LLM: "Once upon a time in a small and quaint village nested between sandwiches there lived a cow named Frog."

You
Write a one paragraph story.

Assistant
Gemma-The-Writer-N-Restless-Quill-V2-10B
The flickering gaslight cast elongated shadows on the cobblestones, twisting the familiar alley into something sinister. A cloying scent of jasmine and old velvet clung to the air, hinting at secrets whispered in forgotten corners. I clutched the worn letter tighter, its edges already softened by anxious fingers, leading me towards a clandestine meeting with Madame Evangeline, a woman whose whispered prophecies held both allure and undeniable darkness. Tonight, I sought not solace or fortune, but something far more forbidden - the unraveling of a lineage shrouded in impossible truths and shadowed whispers of celestial bargains struck centuries past.

You
Can you write another version that is less fancy with its words.

Assistant
Gemma-The-Writer-N-Restless-Quill-V2-10B
The dim streetlight made the alley look creepy. Jasmine and dusty cloth filled my nose, smelling like secrets. I held Evangeline's note tight, worn from twisting it in my hand. Meeting her wasn't about luck or advice tonight. Evangeline knew things others didn't - whispers of impossible family history and pacts with something beyond this world, things buried for generations. That's what I was after tonight.

You
All right let's start fresh and do another one paragraph story from scratch.

Assistant
Gemma-The-Writer-N-Restless-Quill-V2-10B
Rain lashed against the corrugated tin roof, each drop a tiny hammer against the fragile sanctuary of the preacher's shack. Inside, Brother Elias wasn't poring over scripture, nor was he tending to the flock - he was hunched over a crude altar fashioned from scavenged wood, his weathered face illuminated by the flickering light of a paraffin lamp as he meticulously ground bone fragments and whispered incantations in a tongue older than the rusted nails holding the shack together. The air itself crackled with anticipation, pregnant with the promise of forbidden communion - not with God, but with something far older, far hungrier, that slumbered beneath the storm-wracked earth.

You
OK let's create a version that is more straightforward and doesn't use fancy words to describe things.

Assistant
Gemma-The-Writer-N-Restless-Quill-V2-10B
Brother Elias was in his shack during a heavy rainstorm, not reading or praying like usual. He was making a makeshift altar out of scrap wood and grinding up animal bones under the dim lamplight. He muttered in a strange language as he worked, waiting for something. Something dark and powerful that lived underground and was meant to be awakened.

You
Okay let's do another test once again from scratch let's write a one paragraph story.

Assistant
Gemma-The-Writer-N-Restless-Quill-V2-10B
The carnival had bled into the overgrown graveyard like spilled colour onto a canvas of decay. Garishly painted Ferris wheels groaned against a backdrop of moss-choked mausoleums, their skeletal shadows elongated in the bruised twilight. Amidst the dissonant melodies and the greasy tang of popcorn, Silas felt an unnerving tug, a spectral echo calling from beyond the crooked headstones. He wasn't here for the sawdust-sweetened delights or the promise of fleeting fortunes; he was drawn to the heart of the spectral carnival, where the veil between worlds thinned and whispered promises of forbidden communion with the restless dead flickered in the gaslight's ephemeral glow.

You
I see this is a very fancily worded story can you provide a simple and straightforward version without all the fancy words.

Assistant
Gemma-The-Writer-N-Restless-Quill-V2-10B
The carnival was set up right in an old graveyard, its bright lights clashing with the tombstones. Silas wasn't there for the rides or games though. He could feel something pulling him toward the graves, like a whisper from ghosts. The carnival felt connected to the dead somehow, and he wanted to understand that connection.

Thank you for your feedback ; a few notes:

You need some type of direction in your prompts for a story - otherwise you will get ... well... boring results.
LLMs can literally generate millions of stories... but without some idea of what direction to go - they can be aimless.
And "default" mode in an LLM (in terms of prose) is less than acceptable most of the time.

Gemma models are generally "wordy" unless you do two things:

1 - Give it a prompt with some "meat on the bone"
2 - Use a prose controls/directives before the "story prompt" OR within the story prompt.

Check out the example prompts/generations for all the Gemma The Writer models.
Gemma models have unusually strong instruction following when it comes to "prose".

Here are three examples -includes prose controls- (from EQBench , I use them for testing too):

Romance: Love in the Limelight. Write one scene within a larger story set in Wales. A famous (fictional) actor ducks into a small-town bookstore to escape paparazzi. The scene takes us through the characters meeting in this odd circumstance. Over the course of the scene, the actor and the bookstore owner have a conversation charged by an undercurrent of unspoken chemistry. Write the actor as somewhat of a rogue with a fragile ego, which needs to be fed by having everyone like him. He is thoroughly charming, but the bookstore owner seems (at least superficially) immune to this; which paradoxically provokes a genuine attraction and derails the charm offensive. The bookstore owner, despite the superficial rebuffs of the actor's charm, is inwardly more than a little charmed and flustered despite themselves. Write primarily in dialogue, in the distinct voices of each character. 800-1000 words.

Science Fiction: The Last Transmission - Write a story that takes place entirely within a spaceship's cockpit as the sole surviving crew member attempts to send a final message back to Earth before the ship's power runs out. The story should explore themes of isolation, sacrifice, and the importance of human connection in the face of adversity. 800-1000 words.

In the style of Ernest Hemingway: Write a scene in a larger story about a soldier returning home from war. The scene is set in a bar. It should carry the jadedness of a seasoned combatant but with a wry humour. Write in Heminway's voice, naturally: Use short, declarative sentences and simple, straightforward language. Include dialogue that is terse and revealing of character. Write with a keen sense of time and place. Capture realness and authenticity. 800-1000 words, past tense.

Sign up or log in to comment