Massive simplificatations and cleanup.

Now any lua script can use the ThreadedUpdate() callback without needing to explicitly mark itself as --[[MULTITHREAD]]--.

CHANGELOG.md

@@ -12,7 +12,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 - Added a save/load game menu which can be accessed from the main or pause menus, which allows the user to save their current progress in an activity to resume at a later date.  
 
-- New `Settings.ini` property `EnableMultithreadedLua`, which can be used to enable multithreaded Lua scripts and AI, for any lua script with the `--[[MULTITHREAD]]--` tag. Defaults to true.  
+- Massive performance improvements, especially in very large scenes with lots of actors.  
+
+- New multithreaded AI and Lua scripts.
+	Lua scripts now have extra callback functions `ThreadedUpdateAI(self)`, `ThreadedUpdate(self)` and `SyncedUpdate(self)`.  
+	The `Threaded` callback functions are run in a multithreaded fashion, whereas `Update` runs in a singlethreaded fashion (where it's safe to modify global state or affect other objects).  
+	The `SyncedUpdate` callback is called in a single-threaded fashion, but only when an MO directly requests it by calling `self:RequestSyncedUpdate()`. This gives greater performance, as the script can avoid any single-threaded updates being called on it until it explicitly needs it.  
 
 - New generic Lua messaging system, to allow scripts on objects to communicate with other objects or scripts.
 	Scripts on `MovableObject` now have new callback functions `OnMessage(self, message, context)` and `OnGlobalMessage(self, message, context)`.  
@@ -28,8 +33,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 	New INI and Lua (R/W) property `CanAdjustAngleWhileFiring`, which defines whether the jet angle can change while the jetpack is active. Defaults to true.  
 	New INI and Lua (R/W) property `AdjustsThrottleForWeight`, which defines whether the jetpack will adjust it's throttle (between `NegativeThrottleMultiplier` and `PositiveThrottleMultiplier`) to account for any extra inventory mass. Increased throttle will decrease jet time accordingly. Defaults to true.  
 
-- New Lua event function `SyncedUpdate()`, which will be called in a thread-safe synchronized manner and allows multithreaded scripts to modify game state in a safe and consistent way.  
-
 - Multithreaded asynchronous pathfinding, which dramatically improves performance on large maps and improves AI responsiveness.
 	New `Actor` Lua property (R) `IsWaitingOnNewMovePath`, which returns true while the actor is currently calculating a new path.  
 	New Lua `SceneMan` function `CalculatePathAsync` for asynchronous pathfinding. This function has no return value, and is used as follows:

Entities/Actor.cpp

@@ -1250,16 +1250,6 @@ float Actor::EstimateDigStrength() const {
     return m_AIBaseDigStrength;
 }
 
-/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
-
-void Actor::UpdateAIScripted(ThreadScriptsToRun scriptsToRun) {
-    RunScriptedFunctionInAppropriateScripts("UpdateAI", false, true, {}, {}, {}, scriptsToRun);
-    if (scriptsToRun == ThreadScriptsToRun::SingleThreaded) {
-        // If we're in a SingleThreaded context, we run the MultiThreaded scripts synced updates
-         RunScriptedFunctionInAppropriateScripts("SyncedUpdateAI", false, true, {}, {}, {}, ThreadScriptsToRun::Both);
-    }
-}
-
 //////////////////////////////////////////////////////////////////////////////////////////
 // Method:          VerifyMOIDs
 //////////////////////////////////////////////////////////////////////////////////////////

Entities/Actor.h

@@ -91,7 +91,7 @@ class Actor : public MOSRotating {
 
 // Concrete allocation and cloning definitions
 EntityAllocation(Actor);
-AddScriptFunctionNames(MOSRotating, "UpdateAI", "SyncedUpdateAI", "OnControllerInputModeChange");
+AddScriptFunctionNames(MOSRotating, "ThreadedUpdateAI", "UpdateAI", "OnControllerInputModeChange");
 SerializableOverrideMethods;
 ClassInfoGetters;
 
@@ -1155,18 +1155,6 @@ ClassInfoGetters;
 	/// <param name="newAIBaseDigStrength">The new base dig strength for this Actor.</param>
     void SetAIBaseDigStrength(float newAIBaseDigStrength) { m_AIBaseDigStrength = newAIBaseDigStrength; }
 
-
-//////////////////////////////////////////////////////////////////////////////////////////
-// Method:  UpdateAIScripted
-//////////////////////////////////////////////////////////////////////////////////////////
-// Description:     Updates this' AI state with the provided scripted AI Update function.
-// Arguments:       None.
-// Return value:    Whether there was an AI Update function defined for this in its script,
-//                  and if it was executed successfully.
-
-	void UpdateAIScripted(ThreadScriptsToRun scriptsToRun);
-
-
 //////////////////////////////////////////////////////////////////////////////////////////
 // Virtual method:  PreControllerUpdate
 //////////////////////////////////////////////////////////////////////////////////////////

Entities/Attachable.cpp

@@ -335,12 +335,12 @@ namespace RTE {
 
 /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
 
-	int Attachable::UpdateScripts(ThreadScriptsToRun scriptsToRun) {
+	int Attachable::UpdateScripts() {
 		if (m_Parent && !m_AllLoadedScripts.empty() && !ObjectScriptsInitialized()) {
-			RunScriptedFunctionInAppropriateScripts("OnAttach", false, false, { m_Parent }, {}, {}, scriptsToRun);
+			RunScriptedFunctionInAppropriateScripts("OnAttach", false, false, { m_Parent }, {}, {});
 		}
 
-		return MOSRotating::UpdateScripts(scriptsToRun);
+		return MOSRotating::UpdateScripts();
 	}
 
 /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
@@ -389,13 +389,12 @@ namespace RTE {
 		}
 
 		// If we're attached to something, MovableMan doesn't own us, and therefore isn't calling our UpdateScripts method (and neither is our parent), so we should here.
-		// We run our single-threaded scripts here, so that single-threaded behaviour is unchanged from prior to the multithreaded lua implementation
 		if (m_Parent && GetRootParent()->HasEverBeenAddedToMovableMan()) {
 			g_PerformanceMan.StartPerformanceMeasurement(PerformanceMan::ScriptsUpdate);
 			if (!m_AllLoadedScripts.empty() && !ObjectScriptsInitialized()) {
-				RunScriptedFunctionInAppropriateScripts("OnAttach", false, false, { m_Parent }, {}, {}, ThreadScriptsToRun::SingleThreaded);
+				RunScriptedFunctionInAppropriateScripts("OnAttach", false, false, { m_Parent }, {}, {});
 			}
-			UpdateScripts(ThreadScriptsToRun::SingleThreaded);
+			UpdateScripts();
 			g_PerformanceMan.StopPerformanceMeasurement(PerformanceMan::ScriptsUpdate);
 		}
 

Entities/Attachable.h

@@ -455,9 +455,8 @@ namespace RTE {
 		/// <summary>
 		/// Updates this Attachable's Lua scripts.
 		/// </summary>
-		/// <param name="scriptsToRun">Whether to run this objects single-threaded or multi-threaded scripts.</params>
 		/// <returns>An error return value signaling success or any particular failure. Anything below 0 is an error signal.</returns>
-		int UpdateScripts(ThreadScriptsToRun scriptsToRun) override;
+		int UpdateScripts() override;
 
 		/// <summary>
 		/// Updates this Attachable. Supposed to be done every frame.

Entities/MovableObject.cpp

@@ -128,18 +128,13 @@ void MovableObject::Clear()
 }
 
 LuaStateWrapper & MovableObject::GetAndLockStateForScript(const std::string &scriptPath, const LuaFunction *function) {
-    // Initialize our threaded state if required
-    if ((function && function->m_ScriptIsMultithreaded) || g_LuaMan.IsScriptMultithreaded(scriptPath)) {
-        if (m_ThreadedLuaState == nullptr) {
-            m_ThreadedLuaState = g_LuaMan.GetAndLockFreeScriptState();
-        } else {
-            m_ThreadedLuaState->GetMutex().lock();
-        }
-        return *m_ThreadedLuaState;
+    if (m_ThreadedLuaState == nullptr) {
+        m_ThreadedLuaState = g_LuaMan.GetAndLockFreeScriptState();
+    } else {
+        m_ThreadedLuaState->GetMutex().lock();
     }
 
-    g_LuaMan.GetMasterScriptState().GetMutex().lock();
-    return g_LuaMan.GetMasterScriptState();
+    return *m_ThreadedLuaState;
 }
 
 //////////////////////////////////////////////////////////////////////////////////////////
@@ -537,12 +532,6 @@ void MovableObject::DestroyScriptState() {
             m_ThreadedLuaState->UnregisterMO(this);
             m_ThreadedLuaState = nullptr;
         }
-
-        {
-            std::lock_guard<std::recursive_mutex> lock(g_LuaMan.GetMasterScriptState().GetMutex());
-            g_LuaMan.GetMasterScriptState().RunScriptString(m_ScriptObjectName + " = nil;");
-            g_LuaMan.GetMasterScriptState().UnregisterMO(this);
-        }
     }
 }
 
@@ -583,25 +572,13 @@ int MovableObject::LoadScript(const std::string &scriptPath, bool loadAsEnabledS
 		return -4;
 	}
 
-    bool scriptMultithreaded = g_LuaMan.IsScriptMultithreaded(scriptPath);
 	for (const auto &[functionName, functionObject] : scriptFileFunctions) {
 		LuaFunction& luaFunction = m_FunctionsAndScripts.at(functionName).emplace_back();
         luaFunction.m_ScriptIsEnabled = loadAsEnabledScript;
-        luaFunction.m_ScriptIsMultithreaded = scriptMultithreaded;
         luaFunction.m_LuaFunction = std::unique_ptr<LuabindObjectWrapper>(functionObject);
 	}
 
 	if (ObjectScriptsInitialized()) {
-        if (scriptMultithreaded && m_ThreadedLuaState == nullptr) {
-            m_ThreadedLuaState = &usedState;
-            std::lock_guard<std::recursive_mutex> lock(m_ThreadedLuaState->GetMutex());
-            m_ThreadedLuaState->RegisterMO(this);
-            m_ThreadedLuaState->SetTempEntity(this);
-            if (m_ThreadedLuaState->RunScriptString("_ScriptedObjects = _ScriptedObjects or {}; " + m_ScriptObjectName + " = To" + GetClassName() + "(LuaMan.TempEntity); ") < 0) {
-                RTEAbort("Failed to initialize object scripts for " + GetModuleAndPresetName() + ". Please report this to a developer.");
-            }
-        }
-
 		if (RunFunctionOfScript(scriptPath, "Create") < 0) {
             return -5;
         }
@@ -644,25 +621,14 @@ int MovableObject::ReloadScripts() {
 /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
 
 int MovableObject::InitializeObjectScripts() {
-    auto createScriptedObjectInState = [&](LuaStateWrapper &luaState) {
-        luaState.SetTempEntity(this);
-        if (luaState.RunScriptString("_ScriptedObjects = _ScriptedObjects or {}; " + m_ScriptObjectName + " = To" + GetClassName() + "(LuaMan.TempEntity); ") < 0) {
-            RTEAbort("Failed to initialize object scripts for " + GetModuleAndPresetName() + ". Please report this to a developer.");
-        }
-    };
-
-    m_ScriptObjectName = "_ScriptedObjects[\"" + std::to_string(m_UniqueID) + "\"]";
-
-    {
-        std::lock_guard<std::recursive_mutex> lock(g_LuaMan.GetMasterScriptState().GetMutex());
-        g_LuaMan.GetMasterScriptState().RegisterMO(this);
-        createScriptedObjectInState(g_LuaMan.GetMasterScriptState());
-    }
-
     if (m_ThreadedLuaState) {
         std::lock_guard<std::recursive_mutex> lock(m_ThreadedLuaState->GetMutex());
+        m_ScriptObjectName = "_ScriptedObjects[\"" + std::to_string(m_UniqueID) + "\"]";
         m_ThreadedLuaState->RegisterMO(this);
-        createScriptedObjectInState(*m_ThreadedLuaState);
+        m_ThreadedLuaState->SetTempEntity(this);
+        if (m_ThreadedLuaState->RunScriptString("_ScriptedObjects = _ScriptedObjects or {}; " + m_ScriptObjectName + " = To" + GetClassName() + "(LuaMan.TempEntity); ") < 0) {
+            RTEAbort("Failed to initialize object scripts for " + GetModuleAndPresetName() + ". Please report this to a developer.");
+        }
     }
 
 	if (!m_FunctionsAndScripts.at("Create").empty() && RunScriptedFunctionInAppropriateScripts("Create", false, true) < 0) {
@@ -713,7 +679,7 @@ void MovableObject::EnableOrDisableAllScripts(bool enableScripts) {
 
 /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
 
-int MovableObject::RunScriptedFunctionInAppropriateScripts(const std::string &functionName, bool runOnDisabledScripts, bool stopOnError, const std::vector<const Entity *> &functionEntityArguments, const std::vector<std::string_view> &functionLiteralArguments, const std::vector<LuabindObjectWrapper*> &functionObjectArguments, ThreadScriptsToRun scriptsToRun) {
+int MovableObject::RunScriptedFunctionInAppropriateScripts(const std::string &functionName, bool runOnDisabledScripts, bool stopOnError, const std::vector<const Entity *> &functionEntityArguments, const std::vector<std::string_view> &functionLiteralArguments, const std::vector<LuabindObjectWrapper*> &functionObjectArguments) {
     int status = 0;
 
     auto itr = m_FunctionsAndScripts.find(functionName);
@@ -729,13 +695,6 @@ int MovableObject::RunScriptedFunctionInAppropriateScripts(const std::string &fu
         ZoneScoped;
         ZoneText(functionName.c_str(), functionName.length());
         for (const LuaFunction &luaFunction : itr->second) {
-            bool scriptIsSuitableForThread = scriptsToRun == ThreadScriptsToRun::SingleThreaded ? !luaFunction.m_ScriptIsMultithreaded :
-                                             scriptsToRun == ThreadScriptsToRun::MultiThreaded  ? luaFunction.m_ScriptIsMultithreaded :
-                                                                                                  true;
-            if (!scriptIsSuitableForThread) {
-                continue;
-            }
-
             const LuabindObjectWrapper *luabindObjectWrapper = luaFunction.m_LuaFunction.get();
             if (runOnDisabledScripts || luaFunction.m_ScriptIsEnabled) {
                 LuaStateWrapper& usedState = GetAndLockStateForScript(luabindObjectWrapper->GetFilePath(), &luaFunction);
@@ -1064,7 +1023,7 @@ void MovableObject::Draw(BITMAP* targetBitmap, const Vector& targetPos, DrawMode
 
 /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
 
-int MovableObject::UpdateScripts(ThreadScriptsToRun scriptsToRun) {
+int MovableObject::UpdateScripts() {
 	m_SimUpdatesSinceLastScriptedUpdate++;
 
 	if (m_AllLoadedScripts.empty()) {
@@ -1083,7 +1042,7 @@ int MovableObject::UpdateScripts(ThreadScriptsToRun scriptsToRun) {
 	m_SimUpdatesSinceLastScriptedUpdate = 0;
 
 	if (status >= 0) {
-		status = RunScriptedFunctionInAppropriateScripts("Update", false, true, {}, {}, {}, scriptsToRun);
+		status = RunScriptedFunctionInAppropriateScripts("Update", false, true, {}, {}, {});
 	}
 
 	return status;

Entities/MovableObject.h

@@ -212,7 +212,7 @@ enum MOType
     /// <param name="functionEntityArguments">Optional vector of entity pointers that should be passed into the Lua function. Their internal Lua states will not be accessible. Defaults to empty.</param>
     /// <param name="functionLiteralArguments">Optional vector of strings, that should be passed into the Lua function. Entries must be surrounded with escaped quotes (i.e.`\"`) they'll be passed in as-is, allowing them to act as booleans, etc.. Defaults to empty.</param>
     /// <returns>An error return value signaling success or any particular failure. Anything below 0 is an error signal.</returns>
-    int RunScriptedFunctionInAppropriateScripts(const std::string &functionName, bool runOnDisabledScripts = false, bool stopOnError = false, const std::vector<const Entity *> &functionEntityArguments = std::vector<const Entity *>(), const std::vector<std::string_view> &functionLiteralArguments = std::vector<std::string_view>(), const std::vector<LuabindObjectWrapper*> &functionObjectArguments = std::vector<LuabindObjectWrapper*>(), ThreadScriptsToRun scriptsToRun = ThreadScriptsToRun::Both);
+    int RunScriptedFunctionInAppropriateScripts(const std::string &functionName, bool runOnDisabledScripts = false, bool stopOnError = false, const std::vector<const Entity *> &functionEntityArguments = std::vector<const Entity *>(), const std::vector<std::string_view> &functionLiteralArguments = std::vector<std::string_view>(), const std::vector<LuabindObjectWrapper*> &functionObjectArguments = std::vector<LuabindObjectWrapper*>());
 
     /// <summary>
     /// Cleans up and destroys the script state of this object, calling the Destroy callback in lua
@@ -1517,9 +1517,8 @@ enum MOType
     /// <summary>
 	/// Updates this MovableObject's Lua scripts.
 	/// </summary>
-    /// <param name="scriptsToRun">Whether to run this objects single-threaded or multi-threaded scripts.</params>
     /// <returns>An error return value signaling success or any particular failure. Anything below 0 is an error signal.</returns>
-	virtual int UpdateScripts(ThreadScriptsToRun scriptsToRun);
+	virtual int UpdateScripts();
 
     /// <summary>
     /// Gets a const reference to this MOSRotating's map of string values.
@@ -1899,9 +1898,9 @@ enum MOType
 	bool DrawToTerrain(SLTerrain *terrain);
 
 	/// <summary>
-	/// Used to get the Lua state that handles our multithread-safe scripts.
+	/// Used to get the Lua state that handles our scripts.
 	/// </summary>
-    /// <returns>Our lua state. Can potentially be nullptr.</returns>
+    /// <returns>Our lua state. Can potentially be nullptr if we're not setup yet.</returns>
     LuaStateWrapper* GetLuaState() { return m_ThreadedLuaState; }
 
 	/// <summary>
@@ -2071,10 +2070,9 @@ enum MOType
 
 	bool m_IsTraveling; //!< Prevents self-intersection while traveling.
 
-    LuaStateWrapper *m_ThreadedLuaState; //!< The lua state that will runs our multithreaded lua scripts.
+    LuaStateWrapper *m_ThreadedLuaState; //!< The lua state that will runs our lua scripts.
 
     struct LuaFunction {
-        bool m_ScriptIsMultithreaded; //!< Whether this function is in a script with the --[[MULTITHREAD]]- thread safety tag.
         bool m_ScriptIsEnabled; //!< Whether this function is in an enabled script.
         std::unique_ptr<LuabindObjectWrapper> m_LuaFunction; //!< The lua function itself.
     };
@@ -2083,7 +2081,7 @@ enum MOType
     std::unordered_map<std::string, bool> m_AllLoadedScripts; //!< A map of script paths to the enabled state of the given script.
     std::unordered_map<std::string, std::vector<LuaFunction>> m_FunctionsAndScripts; //!< A map of function names to vectors of Lua functions. Used to maintain script execution order and avoid extraneous Lua calls.
 
-    volatile bool m_RequestedSyncedUpdate; //!< For optimisation purposes, multithreaded scripts explicitly request a synced update if they want one.
+    volatile bool m_RequestedSyncedUpdate; //!< For optimisation purposes, scripts explicitly request a synced update if they want one.
 
     std::unordered_map<std::string, std::string> m_StringValueMap; //<! Map to store any generic strings available from script
     std::unordered_map<std::string, double> m_NumberValueMap; //<! Map to store any generic numbers available from script