Jump to content

UHQBot

Forum Bot
 View Profile  See their activity

  • Posts

    43,381

  • Joined

  • Last visited

  • Days Won

    25

 Content Type 

Posts posted by UHQBot

    Wide Open: NVIDIA Accelerates Inference on Meta Llama 3   

    in PC Hardware

    Posted

    NVIDIA today announced optimizations across all its platforms to accelerate Meta Llama 3, the latest generation of the large language model (LLM).

    The open model combined with NVIDIA accelerated computing equips developers, researchers and businesses to innovate responsibly across a wide variety of applications.

    Trained on NVIDIA AI

    Meta engineers trained Llama 3 on a computer cluster packing 24,576 NVIDIA H100 Tensor Core GPUs, linked with an NVIDIA Quantum-2 InfiniBand network. With support from NVIDIA, Meta tuned its network, software and model architectures for its flagship LLM.

    To further advance the state of the art in generative AI, Meta recently described plans to scale its infrastructure to 350,000 H100 GPUs.

    Putting Llama 3 to Work

    Versions of Llama 3, accelerated on NVIDIA GPUs, are available today for use in the cloud, data center, edge and PC.

    From a browser, developers can try Llama 3 at ai.nvidia.com. It’s packaged as an NVIDIA NIM microservice with a standard application programming interface that can be deployed anywhere.

    Businesses can fine-tune Llama 3 with their data using NVIDIA NeMo, an open-source framework for LLMs that’s part of the secure, supported NVIDIA AI Enterprise platform. Custom models can be optimized for inference with NVIDIA TensorRT-LLM and deployed with NVIDIA Triton Inference Server.

    Taking Llama 3 to Devices and PCs

    Llama 3 also runs on NVIDIA Jetson Orin for robotics and edge computing devices, creating interactive agents like those in the Jetson AI Lab.

    What’s more, NVIDIA RTX and GeForce RTX GPUs for workstations and PCs speed inference on Llama 3. These systems give developers a target of more than 100 million NVIDIA-accelerated systems worldwide.

    Get Optimal Performance with Llama 3

    Best practices in deploying an LLM for a chatbot involves a balance of low latency, good reading speed and optimal GPU use to reduce costs.

    Such a service needs to deliver tokens — the rough equivalent of words to an LLM — at about twice a user’s reading speed which is about 10 tokens/second.

    Applying these metrics, a single NVIDIA H200 Tensor Core GPU generated about 3,000 tokens/second — enough to serve about 300 simultaneous users — in an initial test using the version of Llama 3 with 70 billion parameters.

    That means a single NVIDIA HGX server with eight H200 GPUs could deliver 24,000 tokens/second, further optimizing costs by supporting more than 2,400 users at the same time.

    For edge devices, the version of Llama 3 with eight billion parameters generated up to 40 tokens/second on Jetson AGX Orin and 15 tokens/second on Jetson Orin Nano.

    Advancing Community Models

    An active open-source contributor, NVIDIA is committed to optimizing community software that helps users address their toughest challenges. Open-source models also promote AI transparency and let users broadly share work on AI safety and resilience.

    Learn more about how NVIDIA’s AI inference platform, including how NIM, TensorRT-LLM and Triton use state-of-the-art techniques such as low-rank adaptation to accelerate the latest LLMs.

    View the full article

    Up to No Good: ‘No Rest for the Wicked’ Early Access Launches on GeForce NOW

    in PC Hardware

    Posted

    It’s time to get a little wicked. Members can now stream No Rest for the Wicked from the cloud.

    It leads six new games joining the GeForce NOW library of more than 1,500 games.

    Holy Moly

    No Rest For The Wicked on GeForce NOWThere’s always another fight to be won.

    No Rest for the Wicked is the highly anticipated action role-playing game from Moon Studios, developer of the Ori series, and publisher Private Division. Amid a plague-ridden world, step into the boots of a Cerim, a holy warrior on a desperate mission. The Great Pestilence has ravaged the land of Sacra, and a new king reigns. As a colonialist inquisition unfolds, engage in visceral combat, battle plague-infested creatures and uncover the secrets of the continent. Make the character you want with the game’s flexible soft-class system, explore a rich storyline, and prepare for intense boss battles as you build up the town of Sacrament.

    Embark on a dark and perilous journey, where no rest awaits the wicked. Rise to the challenge and stream from GeForce RTX 4080 servers with a GeForce NOW Ultimate membership for the smoothest gameplay from the cloud. Be among the first to experience early access of the game, without having to wait for downloads.

    Shiny New Games

    Evil West on GeForce NOW“Yippie ki-yay, evil doers!”

    Become a Wild West superhero in Evil West, streaming on GeForce NOW this week and part of PC Game Pass. It’s part of six newly supported games this week:

    • Kill It With Fire 2 (New release on Steam, April 16)
    • The Crew Motorfest (New release on Steam, April 18)
    • No Rest for the Wicked (New release on Steam, April 18)
    • Evil West (Xbox, available on PC Game Pass)
    • Lightyear Frontier (Steam)
    • Tomb Raider I-III Remastered (Steam)

    Riot Games shared in its 14.8 patch notes that it will soon add its Vanguard security software to League of Legends as part of the publisher’s commitment to remove scripters, bots and bot-leveled accounts from the game and make it more challenging for them to continue. Since Vanguard won’t support virtual machines when it’s added to League of Legends, the game will be put under maintenance and will no longer be playable on GeForce NOW once the 14.9 update goes live globally — currently planned for May 1, 2024. Members can continue to enjoy the game on GeForce NOW until then.

    What are you planning to play this weekend? Let us know on X or in the comments below.

    View the full article

    Patch Notes - April 18, 2024

    in ArcheAge

    Posted

    Patch Notes
    Content Updates
    [Content - Improvements]

    The "Exalted Hiram Infusion" and "Dimensional Hiram Infusion" can now be used on all Hiram Guardian Equipment. 
    • To ease the difficulty of upgrading to Brilliant Hiram Guardian Equipment from Hiram Guardian Equipment, we have expanded the use of "Exalted Hiram Infusion" and "Dimensional Hiram Infusion" from Glorious Hiram Guardian Equipment to all tiers of Hiram Guardian Equipment.
     

    "Enhance" is now available for Erenor and Awakened Erenor equipment. 
    • Based on your feedback about the inability to use the "Enhance" system on Erenor and Awakened Erenor equipment despite their enhanced stats, we have adjusted the system. The "Enhance" feature is now available for both Erenor and Awakened Erenor equipment, ensuring better balance with other equipment.
    • Enhance levels vary depending on the Equipment grade.
    • Weapons
      • Mythic (Lv11) Grade: up to 4 levels
      • Eternal (Lv12) Grade: up to 10 levels
    • Armor
      • Mythic (Lv11) Grade: up to 2 levels
      • Eternal (Lv12) Grade: up to 5 levels
     

    [Abyssal Kraken]

    Instead of awakening during the "Defeat the Kraken!" quest, the Abyssal Kraken will now appear directly with a certain chance.
    • The Abyssal Kraken now has a certain chance to appear when you encounter the Kraken.
     

    [Design]

    Changed the crafting materials for the Luna Charm Rank 6.
    • Removed the "Territory Coin" crafting material.
    • Increased the required quantity of "Superior Lunarite" by 5 (from 20 to 25).
     
    Bug Fixes
    Fixed an issue where Kraken was not displayed on the Sea Monster Sonar.  

    Fixed an issue where Brown Thorn Hedgehog was abnormally spawning in the Great Prairie of the West area. 

     
    Events
    "Garden Arcane Manastone" event has begun. (Until May. 16th (Thur), 2024 before maintenance)  
    • Defeat Legendary boss monsters in Garden of the Gods to receive "Garden Arcane Manastone Crystal" and "Unstable Erenor Infusions" as additional rewards.
    • Complete "Fairy Request" quest at Rank 4 or above to receive "Garden Arcane Manastone" as an additional reward.
    • You can use a "Garden Arcane Manastone" to craft "Sealed Mana Arcane Infusion" with Radiant Hiram Infusion x10.
    • "Sealed Mana Arcane Infusion" can be traded.

    View the full article

×
×
  • Create New...

Important Information

By using this site, you agree to our Guidelines Privacy Policy.

  I accept