Life of an inference request (vLLM V1): How LLMs are served efficiently at scale

Link: ubicloud.com/blog/life-of-an-i…
Discussion: news.ycombinator.com/item?id=4…

Life of an inference request (vLLM V1): How LLMs are served efficiently at scale

vLLM is an open-source inference engine that serves large language models. We deploy vLLM across GPUs and load open weight models like Llama 4 into it.

^{www.ubicloud.com}

Memphis Slim--Only Fools Have Fun:

youtube.com/watch?v=KvAQ7pzbvq…

#musicatlantis

Only Fools Have Fun

Provided to YouTube by Universal Music GroupOnly Fools Have Fun · Memphis SlimThe Sonet Blues Story℗ 1973 Universal Music ABReleased on: 1973-01-01Producer: ...

^YouTube

Dear #Google, thanks for suggesting #TruthSocial while I'm downloading #Fedilab................

#FuckGoogle

Sensitive content Reveal/hide

Soy Melson is especially ugly this evening:

noauthority.social/@SiRrogueKn…

The Barbarity of Solitary Confinement in South Carolina: Kevin Rashid Johnson

Both the United Nations and Courts all over have recognized that continuous confinement inside a cell for even brief periods is torture and has proven in extensive psychological studies to cause mental illness and organic brain damage.

There has been a broad outside and inside movement to reform and end solitary confinement in my home state of Virginia for over a decade. This struggle has led to laws being passed that require daily out-of-cell time for Va prisoners in solitary (1), and a pending federal class action lawsuit in which the presiding judge and the appeals court have found Va’s conditions in solitary to be unconstitutional and harmful to prisoners’ mental health. (2) Not so in South Carolina, where the conditions in solitary are far worse than anything in Va, and where I was recently
interstate transferred.

Not only are the conditions in SC’s solitary confinement units (euphemistically called “Restrictive Housing Units”) downright medieval, but there are no efforts to reform nor end them. Worse still, the corruption and retribution of SC administrative officials operate to stomp out any efforts by prisoners to bring attention to SC’s prison conditions.

I’ve been in SC since May 1, 2025 and have been thrown in solitary twice already, and have gotten to see and experience the barbarity of these units.

Although I was in general population when I left Va, I was thrown in solitary as soon as I arrived in SC. I was initially taken to SC’s Reception and Orientation institution called Kirkland. There I was thrown into a cold solitary cell with nothing but a filthy mattress and no clothes except a nylon smock. Not even underwear. When I protested that I would not accept meals and requested to call attorneys. I was transferred the next day to Perry Correctional (sic!) Institution, where I was again thrown in solitary. Approximately half of the buildings at Perry are solitary confinement units and prisoners are sent to Perry from all over the state to be held in solitary long term.

Here one is confined in a cold cell with a concrete slab raised about 6 inches from the floor. At night when the cells are left completely dark, bugs, spiders and mice crawl on and around one’s body. Prisoners have been held in solitary at Perry for years. My neighbor, Jimmy Causey, has been here for eight years.

There is no view outside the cells. We receive no outside exercise or fresh air. WHen I arrived at Perry on May 2d, the prisoners in solitary had been to outside exercise only five times this entire year. The denial of regular outside exercise in solitary standing alone, has been found unconstitutional by federal courts. While in other states solitary is characterized as 23/1 hours of in-cell confinement, in SC it is literally 24/7. The only times prisoners come out of their cells here is to shower for 15 minutes three times per week, which is often denied as well.

We can have no personal property, no personal books to read, no periodicals. We can’t make canteen purchases. Not even personal hygiene items. The only hygiene items we may use are state-issued toothpaste, toothbrush and soap. No deodorant, shampoo, or hair grease is allowed. One can’t groom or shave, and even could we there are no mirrors to see oneself. Yet the rules say our hair must be kept closely cropped.

We cannot purchase stationary to write loved ones. Only two outgoing letters are permitted per month. We are not allowed to communicate directly with the media. We cannot write to but one lawyer, and only if they are confirmed to be actively representing us in pending legal proceedings. These obstacles to communication obviously work to sever community and family ties in outright contradiction of any claims to rehabilitation and enabling anyone to reenter society as a healthy and productive member. And they prevent public exposure and accountability for the inhumane conditions in these prisons and units, and any prisoner from obtaining attorneys to assist them, whether to challenge discrepancies in their original criminal cases or to contest the conditions of their confinement (the latter being the obvious purpose of these illegal obstructions to communicate with lawyers).

The meals are disgusting and bland, non-nutritious (composed primarily of breads and pasta), repetitious and the portions insufficient. EVERYONE complains of constant hunger pangs and substantial weight loss. Not only are the general portions tiny, but as opposed to the three meals given on weekdays, only two meals are served on weekends. The two weekend meals are basically the same portions given on two of the weekday trays. Worse still, the time between the last meal and the meal the next day on the weekends is approximately 20 hours. This is illegal and literal torture. In fact courts have held that the dull continuous pain of prolonged hunger constitutes corporal punishment.

So prisoners in SC’s solitary sit idle, stripped of all fresh air, sensory stimulation, and diversionary activities day after day, for months to years on end, under the most blatant conditions of mental torture. I’ve not seen this extreme level of total denial of environmental stimulation and solitary confinement anywhere else in my 35 years of confinement. In fact, this sort of solitary was outlawed by federal courts in Va over 50 years in the landmark case of Landman v. Royster. (3)

Not only are the solitary conditions inhumane per se, but they’re coupled with physical abuses sanctioned by administrators. Prisoners are routinely gassed and beaten by guards. They’re hogtied and left naked in empty cells for hours on end. Of course hogtying has often proved fatal and has been banned in prisons for decades. (4)

I’ve observed others left in empty cells to sleep on bare concrete with no clothes except boxers for three days, for nothing but making noise by striking a cell door with their hands. In one case the prisoner only wanted to be taken outside so he could perform his wiccan prayers after months of prisoners in solitary illegally receiving no outside exercise.

To add greater insult to the mistreatment, food, medications, and showers are illegally used as punishment and to compel submission. One is denied meals, meds, and showers if they are not fully dressed and standing in the rear of their cell when guards arrive at their cells to provide these things. Prisoners are not required to be dressed and standing in this manner at any other times when guards come to their cells. The measure is calculated to use needed essentials as weapons and tools of forced compliance with arbitrary orders to instill docility and submission. Like training animals.. Or breaking slaves.

Suffering under these conditions for just days is literally torture. As it’s meant to be. SC’s prisoners need a support movement to end this abuse.

Dare to struggle Dare to Win!

All Power to the People!
———————————–

Endnotes:
1. See, for example, Va. Code, section 53.1-39.2 (limitations on
Restorative housing units).
2. Thorpe v. Clarke, 37 F. 4th 926 (4th Cir. 2022)
3. Landman v. Royster, 333 F. Supp. 621 (E.D.va <e.d.va&gt;. 1971)
4. Weigel v. Broad, 544 F. 3d 1143 (coth cim. 2008)

source: Kevin Rashid

abolitionmedia.noblogs.org/?p=…

#blackLiberation #kevinRashidJohnson #newAfrikanPantherParty #northAmerica #repression #southCarolina

Uh-Oh! Political Antisemitism Smears Have Stopped Working!

Zohran Mamdani has won the Democratic Party primary in the New York City mayoral race after a frenetic smear campaign to portray him as an antisemite for his pro-Palestinian views, and now all the world’s worst people are freaking out about it.

caitlinjohnst.one/p/uh-oh-poli…

Uh-Oh! Political Antisemitism Smears Have Stopped Working!

Notes From The Edge Of The Narrative Matrix

^{Caitlin Johnstone (Caitlin’s Newsletter)}

Lovely write-up of the wealth of housing co-ops in Liverpool from the Liverpool Post

livpost.co.uk/liverpool-has-th…

#Liverpool #housing #cooperative

Liverpool has the most cooperative housing outside of London. So why does no one know about them?

Ella Benson Easton journeys through the city’s co-ops to find out why this radical way of living is one of Liverpool’s best kept secrets

^{Ella Benson Easton (The Post)}

White people today: the protesters in LA are hurting their own cause, they should be like MLK and the Civil Rights Movement!

Meanwhile, white people during the Civil Rights Movement:

Jeff Daniels does a good job playing Chamberlain in the movie "Gettysburg." And, I admit I don't know much about Chamberlain, he does seem to be a good man who fought one hell of a fight during that battle.

But the speech he gives at the beginning of the movie is such horse shit. Perhaps he sincerely believed it, but it's absolute nonsense. And the brutality and raping and pillaging of Yankee troops against Southern civilians time after time after time is evidence against these lies.

This got a little Dodge-y at the end (no pun intended)

x.com/Tr00peRR/status/19391339…

Source: libsoftiktok

Suing woke companies is sometimes the only way regular Americans can fight back.

Remember those companies that discriminated against White or straight people? Or forced employees to use pronouns??

Most Americans have no way of fighting back without help from outside sources.

Why cut that off?

xcancel.com/DC_Draino/status/1…

Original tweet : xcancel.com/libsoftiktok/statu…

Sensitive content Reveal/hide

Female nudity, erotics, cute and sexy

#nsfw #sexy #ero #erotic

This is GOLD

noauthority.social/@bot@nicecr…

#FaggotTourney